{
 "metadata": {
  "name": "",
  "signature": "sha256:a95598b0e0a09d27ea5633ad46db5bfeb206e278e65d9c557481b2b21b1c7aa5"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Hands-on with PyData: How to Build a Minimal Recommendation Engine"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Welcome!\n",
      "\n",
      "- About Unata\n",
      "    - What we do\n",
      "    - What we use Python for (everything!)\n",
      "- Environment + data files check\n",
      "    - Local Setup\n",
      "        - This is the _prefered setup_.\n",
      "        - Instructions to set up a local environment and links to download handout and data files: [http://unatainc.github.io/pycon2014/](http://unatainc.github.io/pycon2014/)\n",
      "    - Hosted Setup with Ipydra:\n",
      "        - This is only a _fallback option_ for those without a local environment.\n",
      "        - Hosted environment: [http://pycon.unata.com](http://pycon.unata.com)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## The Recommendation Problem\n",
      "\n",
      "Recommenders have been around since at least 1992. Today we see different flavours of recommenders, deployed across different verticals: \n",
      "\n",
      "- Amazon\n",
      "- Netflix\n",
      "- Facebook\n",
      "- Last.fm.\n",
      "\n",
      "What exactly do they do?\n",
      "\n",
      "### Definitions from the literature\n",
      "      \n",
      "- *In a typical recommender system people provide recommendations as inputs, which\n",
      "the system then aggregates and directs to appropriate recipients.* -- Resnick\n",
      "and Varian, 1997\n",
      "\n",
      "- *Collaborative filtering simply means that people collaborate to help one\n",
      "another perform filtering by recording their reactions to documents they read.*\n",
      "-- Goldberg et al, 1992\n",
      "\n",
      "- *In its most common formulation, the recommendation problem is reduced to the\n",
      "problem of estimating ratings for the items that have not been seen by a\n",
      "user. Intuitively, this estimation is usually based on the ratings given by this\n",
      "user to other items and on some other information [...] Once we can estimate\n",
      "ratings for the yet unrated items, we can recommend to the user the item(s) with\n",
      "the highest estimated rating(s).* -- Adomavicius and Tuzhilin, 2005\n",
      "\n",
      "- *Driven by computer algorithms, recommenders help consumers\n",
      "by selecting products they will probably like and might buy\n",
      "based on their browsing, searches, purchases, and preferences.* -- Konstan and Riedl, 2012"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Notation\n",
      "\n",
      "- $U$ is the set of users in our domain. Its size is $|U|$.\n",
      "- $I$ is the set of items in our domain. Its size is $|I|$.\n",
      "- $I(u)$ is the set of items that user $u$ has rated.\n",
      "- $-I(u)$ is the complement of $I(u)$ i.e., the set of items not yet seen by user $u$.\n",
      "- $U(i)$ is the set of users that have rated item $i$.\n",
      "- $-U(i)$ is the complement of $U(i)$.\n",
      "- $S(u,i)$ is a function that measures the utility of item $i$ for user $u$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Goal of a recommendation system\n",
      "\n",
      "$\n",
      "\\newcommand{\\argmax}{\\mathop{\\rm argmax}\\nolimits}\n",
      "i^{*} = \\argmax_{i \\in -I(u)} S(u,i), \\forall{u \\in U}\n",
      "$"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Problem statement\n",
      "\n",
      "The recommendation problem in its most basic form is quite simple to define:\n",
      "\n",
      "```\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "| user_id, movie_id | m_1 | m_2 | m_3 | m_4 | m_5 |\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "| u_1               | ?   | ?   | 4   | ?   | 1   |\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "| u_2               | 3   | ?   | ?   | 2   | 2   |\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "| u_3               | 3   | ?   | ?   | ?   | ?   |\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "| u_4               | ?   | 1   | 2   | 1   | 1   |\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "| u_5               | ?   | ?   | ?   | ?   | ?   |\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "| u_6               | 2   | ?   | 2   | ?   | ?   |\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "| u_7               | ?   | ?   | ?   | ?   | ?   |\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "| u_8               | 3   | 1   | 5   | ?   | ?   |\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "| u_9               | ?   | ?   | ?   | ?   | 2   |\n",
      "|-------------------+-----+-----+-----+-----+-----|\n",
      "```\n",
      "\n",
      "*Given a partially filled matrix of ratings ($|U|x|I|$), estimate the missing values.*\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Well-known Solutions to the Recommendation Problem"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Content-based filtering\n",
      "\n",
      "*Recommend based on the user's rating history.* \n",
      "\n",
      "Generic expression (notice how this is kind of a 'row-based' approach):\n",
      "\n",
      "$$ \n",
      "\\newcommand{\\aggr}{\\mathop{\\rm aggr}\\nolimits}\n",
      "r_{u,i} = \\aggr_{i' \\in I(u)} [r_{u,i'}]\n",
      "$$\n",
      "\n",
      "\n",
      "A simple example using the mean as an aggregation function:\n",
      "\n",
      "$$ \n",
      "r_{u,i} = \\bar r_u = \\frac{\\sum_{i' \\in I(u)} r_{u,i'}}{|I(u)|} \n",
      "$$\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Collaborative filtering\n",
      "\n",
      "*Recommend based on other user's rating histories.* \n",
      "\n",
      "Generic expression (notice how this is kind of a 'col-based' approach):\n",
      "\n",
      "$$ \n",
      "\\newcommand{\\aggr}{\\mathop{\\rm aggr}\\nolimits}\n",
      "r_{u,i} = \\aggr_{u' \\in U(i)} [r_{u',i}] \n",
      "$$"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "A simple example using the mean as an aggregation function:\n",
      "\n",
      "$$ \n",
      "r_{u,i} = \\bar r_i = \\frac{\\sum_{u' \\in U(i)} r_{u',i}}{|U(i)|} \n",
      "$$\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Hybrid solutions\n",
      "\n",
      "The literature has lots of examples of systems that try to combine the strengths\n",
      "of the two main approaches. This can be done in a number of ways:\n",
      "\n",
      "- Combine the predictions of a content-based system and a collaborative system.\n",
      "- Incorporate content-based techniques into a collaborative approach.\n",
      "- Incorporarte collaborative techniques into a content-based approach.\n",
      "- Unifying model."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Challenges\n",
      "\n",
      "#### Availability of item metadata\n",
      "\n",
      "Content-based techniques are limited by the amount of metadata that is available\n",
      "to describe an item. There are domains in which feature extraction methods are\n",
      "expensive or time consuming, e.g., processing multimedia data such as graphics,\n",
      "audio/video streams. In the context of grocery items for example, it's often the\n",
      "case that item information is only partial or completely missing. Examples\n",
      "include:\n",
      "\n",
      "- Ingredients\n",
      "- Nutrition facts\n",
      "- Brand\n",
      "- Description\n",
      "- County of origin\n",
      "\n",
      "#### New user problem\n",
      "\n",
      "A user has to have rated a sufficient number of items before a recommender\n",
      "system can have a good idea of what their preferences are. In a content-based\n",
      "system, the aggregation function needs ratings to aggregate.\n",
      "\n",
      "#### New item problem\n",
      "\n",
      "Collaborative filters rely on an item being rated by many users to compute\n",
      "aggregates of those ratings. Think of this as the exact counterpart of the new\n",
      "user problem for content-based systems.\n",
      "\n",
      "#### Data sparsity\n",
      "\n",
      "When looking at the more general versions of content-based and collaborative\n",
      "systems, the success of the recommender system depends on the availability of a\n",
      "critical mass of user/item iteractions. We get a first glance at the data\n",
      "sparsity problem by quantifying the ratio of existing ratings vs $|U|x|I|$. A\n",
      "highly sparse matrix of interactions makes it difficult to compute similarities\n",
      "between users and items. As an example, for a user whose tastes are unusual\n",
      "compared to the rest of the population, there will not be any other users who\n",
      "are particularly similar, leading to poor recommendations.\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Flow Chart: a Recommendation Engine in Context"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from IPython.core.display import Image \n",
      "Image(filename='./pycon_reco_flow.png')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAvYAAAM2CAYAAACdfscgAACth0lEQVR42uy9d7AdxbW3ff+3XVBl\nrsu4bIOxjW1uvTbX5Cww0USBSSJIJBtjgsjBCJFzTkJCApFB5JxzDhJIJAMmmxyds+etp79vzdtn\nzo4n7n3O81StOmdP6JnpXt3zm57VPf9ViIiIiIhI1/NfZoGIiIiIiMJeREREREQU9iIiIiIiorAX\nERERERGFvYiIiIiIwl5ERERERBT2IiIiIiKisBcREREREYW9iIiIiIjCXkREREREFPYiIiIiIqKw\nFxERERERhb2IiIiIiMJeREREREQU9iIiIiIiorAXERERERGFvYiIiIiIwl5ERERERBT2IiIiIiKi\nsBcRERERke4R9v/1X/+laZqmaZqmaaPGFPaapmmapmmaprDvfGEvIiIiIjKiw2YU9iIiIiIiCnuF\nvYiIiIiIwl5hLyIiIiKisFfYi7TEIYccUneAzQ9/+MNi9913L5599lkzahD561//Wiy11FLFFlts\nUfz73/9ue/977rmneOeddwYkrcHwo7DBOqdOvOaRXndyn6vHlVde2dQnnnnmmWErPxGFvcJeZMSx\nyy67pHrwpS99qVhrrbV6WH4DfvLJJ82sQeLPf/5zsdhii/VJ2MyaNSuVzyOPPFKK3EUWWaRYaaWV\nin/9619D7kfDJeyH85pHW92p+lx/hX1//F9EFPYKe5GM3Xbbra74QDCddNJJaf24ceO86Q4S//jH\nP1IPb1+EzXXXXVcKpOAPf/hD8fvf/35Y/OjRRx8dljwczmsebXWnls81EvbNHmzIq776v4go7BX2\nIi2KE3j77bfT+vXXX79Hb+hrr71W7LrrrsWPf/zjZDvssEPx4osv9tp/7ty5xb777ltuN3HixF7b\nPf/8873Seu6558r1//znP4tJkyYVU6ZMKe699950Lmy37rrrFg8//HBaf8QRRxRf//rX0/LDDjus\n+NOf/tTvfVu91vwYTzzxRLHaaqslobLGGmsU5513Xi+xQhjDhhtumNJaZ511Ug8oPc5VYXPbbbcV\nO++8c3lctiU9jgfTp08vll122VQ+rD/jjDPSQ8Lhhx9eHH/88T3SaiePW7mGdv0opy/5tc0226Tz\n5npvuummYurUqcXee++dRCHp5dfcbvqt+vJQ150HHnggXfdXvvKVtB3+e+655xbHHntseZ0HHXRQ\n+TvPX5ZXfaCV43766afFMccck+pH+Bx5/be//a2uz/3nP/8ZFGHfyGfZ58ADD0zXGfUB7rvvvmKn\nnXYqrr322nIZ53fkkUembUUU9gp7kVEh7Ov1vp111lmlOImb7kMPPVTWn7j5x+/bb7+93BchEssR\nVyussEL5O27Q3IDrbXP11VenbeJVfSwn9AGxU32tz7nE/zxM9HffVq+1eoxqeoif4IorriiXb7LJ\nJj3OJRc2p5xySrl800037XFcRApwnvkxt99+++KPf/xjOpdcTPYljxtdQ3+F/UDlF/8jRCO9uOZ2\n0m/Vl4e67uTXPWHChPTgF7+r11l9cKi1vFU/JtY98nbrrbcut0FUI5Br+Vy9h74Q9s169msJ+2Y+\ny7ngD/x+9dVXy7QiPCrP8/fee6+sXyIKe4W9yKgQ9oiHadOmJdGDnXDCCT1upvQexs2fm3BViIRw\noOebsAh6vUNcXXPNNeV2F154YVpGr9onn3xSCjV6YQPSDXHx4Ycfpht/pDV58uQkVrixH3XUUeV2\n8+bNS/u+/PLLaRlC6C9/+Uu/9m31WvNj7L///sXf//73sqeZZfR+clwERlzvgw8+mLahJ3TPPffs\nIUYQq3FunFMwe/bsHunl4ilCYKoiibTazeNG19DMj0iPY1cNX+LtTZxjq/kV/kee5739ubCvXnOr\n6bdavkNdd3I/eeyxx8rr3muvvXr4Sb2e7uryVo9L7z2/99tvv3IbfIP6wAMxeV3L5+oR2yHMqe+5\ncQ30yPfHZyP9eDjN25zwDbj55pt7bCeisFfYi4x4YV/PCF94/PHHy+1feeWVHuIIi1fh0VtGr22I\nhOjpy0XHcccdl8JMELfVHvLg0EMPLQfoxY0/buhB3LCJZQ6iJ68q+vqyb6vXWu8Y1dh5HiDYh9lU\ncj766KNSELMdseKcX4h/4Nhvvvlmr5CdarhDVSSFeOtrHrca/9/Mj6oDfFs5FqFS8UCWQ/6QD42E\nfSvpt1q+Q113XnjhhZb8pFVh326d5Ri33HJLOl4cN/6v5XPNhH2r/tCuz7711ls92pioX2Hx4ME1\n8mDSbBYfEYW9wl5kxAj7yy67rHj99dfTzf20007rFaoRXH/99U0FHDf8OXPmpP+JCW5247/hhht6\nrYtX8YQzxI2fnsXohcuFEttUr6kq+vqyb6vXGsdAbBIKU09gRXhFNU8RWfTw5uEDQE8vvbTVWVby\n7ZoJ+3bzuNk1NPOju+++O72FoPe0aiEmWz3WJZdc0qPHu1kZVwVvs/RbLd+hrjtRZjNnzuyxfzx4\ntttj3+pxKR/GflRDmBiXwNu1vgp7HuJ5GPv44497WNUf2vVZth0zZkyq2/gc54lPxKw9Z555Znrz\nhh80e+MkorBX2IuMKGFfjYPNY1zzV+4htrADDjggDWDDGMjIAEb+Ej5y5513pm0IUahH9Jrff//9\ndUUBDwb5jT+/Obcj7Puyb6vX2qrAivQQOlXBNn78+B6CLQ/loLeRWGYGQ1Zj8ZsJ+77kcaNraOZH\nrcTYt3qsOL++CvtWy6NZ+Q513YntLrjggprHbFfYt3udPJwxaDcX+OQ1PeR9EfZPPfVUW/7Qqs8C\nA6T5zSBp6hBTnn7++edletH7X31IElHYK+xFRrSwr3WTjoFy9HhFrHG87iZ2uQqxwcSB00sWwrka\nTsCNm5leiLO96qqr0jbEu1c5/fTTe/WIVwVMu8K+3X1bvdZWBVa8xaheL73ZeYhNiBF66vPwAdZV\nj9OqsO9rHg+nsI9zr4biRCx1f4V9q+U71HUHkZoPks7PP5/vPX5Xe6Or27V6XPLyrrvuKnvn6QV/\n+umny0Gq0YPerrBvd1acVn0WInwoQujCV6phUvkAWxGFvcJeZFQKe270MRtHCHRilrmB5oNOgRjc\n2JYezNiO3mZihoMQLcztjWiN3sCXXnqph+iOwXPMnjNcwr7Va21VSEZMcHVQ7EUXXdQjxCZ6WPP4\nf2CqQZbTmx9hDM0Gz8YA1L7mcbvCPuKmOb+qhfhs9VgxiBIfiuuLqQubDZ5tJf1Wy3eo605+3Xnd\nufzyy2sOnmVZDETN/SS2a/W4MT99NbY9esWrwr7VwbPtCvtWfTb2jZl8IhQM8vAjevHzKTFFFPYK\ne5ERS7NBghFSk29z9tlnl8voIWMmkLjh5h/jCUGAMX1jzEST34CZMzuW8f+JJ57Ya1rHel+mbEWc\n92ffVq+13jFqLY/0EG3kSd6zGNtFLyTGYEFitceOHdsjLCIGheZhFrvvvns53WV+zP7kcatfBW3l\ny7MYM+O0c6x46Ile2Xzax3pl3JfyaObLQ113ZsyY0aPu5NNM5ucfg0nxJ8o295M8/KyV4+az0ZDX\n559/fppHPtKPUJyqzzWb7rKZsK9VLq34bBDbcu7vvvtuWhYP0bXquIjCXmEvMmLh9Xyjnkl6SGvN\nDc0UltX54InXrX7cKcJtcjGG4Mlh8GG+DSICYRPHil65euKcKTSr4jxCIPqzb9DsWusdo9Zy8pP4\n4DwtHngQUvl2iPlqvhG7HVNjhlhCwITYDWFT61z6msf1llehV7oVYR9vB9o5FqFJ+TVy3vxddNFF\ny1CofL9202/Vl4e67kQPfdhmm23WS9gjiglry7fDp3/+85/36Tp58GJAar4NeZb39Fd9Lh+UXkvY\ntzKPfV98NogQtzwkKQakV99miCjsFfYiUgduoh988EGa4aJRLDLziNPDzHb1vlLJNqRFfG8nvjZv\n9VpbhfjlmB2kWb6RJ/XyLchnnWmUXifncRVCMnhgQLhVrxVhWZ1FqJPKd6DrDtfcaOpRZp0hvXwW\noP4cF7HONp999lm/fK6/dJvPiijsFfYiIlJH2EevMLHmCDsEXry1qA6qHem0Ot5BRERhLyIiHUe9\nEB967OuFgIxU4k0Fg0Gdk11EFPYKexGRroPZVyZNmpR66plznRlPiC0fbRCOxZiVm266qWloloiI\nwl5ERERERGGvsBcRERERUdgr7EVEREREFPYKexERERERhb2IiIiIiMK+24W9pmmapmmapo0GU9hr\nmqZpmqZpmsLeUBwR/VdE3xX9V8RQHCuniP4r+q6I/iuisLdyihVURN8V0X9F31XYj1r+8pe/FO+/\n/37xzjvvFH/605/MEG8uIvquiP4r+q7Cvpt4+eWXi1133bXXQIt99923+OCDD8rt/vrXvxZLLbVU\nscUWWxT//ve/e/0eLO655570sCH6b195+OGHU7595StfKd59992a2+y///41Bxx96UtfKjbbbLPi\nyiuv7OHnN998c1o/c+ZMM7iLffeQQw6pO9jshz/8YbH77rsXzz77bEfmaaNzDxus9nmo2n/9d+Cp\n19ZRnnvuuWfx9NNPD/k9e7T5k8LeyjloXHPNNWWeIHoOOOCAYq+99uohahD+8Oc//7lYbLHFegj7\nRRZZpFhppZWKf/3rX4NyfrNmzUrn8cgjj1hY+m+f+M9//lPssssuZd5Nmzat5nb5NvWMh13SA4Q+\ny8466ywzuYt9N8qdtm6ttdbqYXnZP/nkkx177sMl7Ae7/dd/h89v+tqu9fWeXdUXorBXGPWBOXPm\nlDc0nrBDsMDf/va34sgjj0zrabj/+c9/poacHqy84v3hD38ofv/73w/aOV533XXpHJ555hkLTP/t\nE++9916PGxZihLCzKrvttlta/+ijj/Zah6jjwZf1c+fOVdiPIN+Ncq8l3GnzTjrppLR+3LhxHSc4\nGvnsUDDY7b/+O7R+g7i+4YYbymu9++67h+yeHfpi/fXX76FFRGGvMGoRKs4mm2yS8uKyyy5rWNEQ\nNPTaV4U9Yv/www8vjj/++B43vNdeey2F9vz4xz9OtsMOOxQvvvhiuZ79Jk2aVEyZMqV44oknitVW\nWy2lu8YaaxTnnXdemdb06dOLZZddNp0j6ZxxxhlWUP23bWbMmJHy7NZbby1OOeWU9P9dd93VlsCD\niRMn9livsB/5wh7efvvttB7BkfdMN2vnAh4EedMT2+FH1e2ef/75Xmk999xz/T73nFbb3YDOnm22\n2SadD+3wTTfdVEydOrXYe++9072g2v63m36r+af/Dr3PX3/99Wn9CiuskMo1uO2224qdd965LLN1\n1lknlW1sU+ueHSK92b6hLwh7xPfoUAy9ce+99/Y6x2bpwaefflocc8wxxbrrrltugw/TcdnNvqiw\nVxjVhF4Wei4R7Th/PfIKUBX28eosv+E99NBDZR5HRYrft99+e9krwH55Lyrbxv8h4LkZ5ttsv/32\niiP9t+1eIOI28fPPP/88Nda1RFp+s6vV08TNacMNN+zRy6WwH1nCvl4PI+UbPhPitJV2DhAksRyh\ni1CK3yHcr7322rrbXH311QMm7Fttd+GKK64ol9MBFG+rImSTe0a1/W8n/VbzT/8dHmHP5BlVfRCd\nItimm27ao8x4u1/vnk2daWXf0Be5nyHI43feAdlKevgjbX+ktfXWW5fbINzjgaMbfVFhrzCqSQgc\nKk7+hNtMJOXCvpbQj4qZV4ioOF//+tfTq1v2ixsAg3j+/ve/lz1EcU4hukI8DderZm8u3c3jjz9e\nxsYDfhrCiV7SWje7G2+8MT0EMHD8448/Ll566aU0gLI6+FZhP7KE/YQJE9L4CwQodsIJJ/QQ2Q88\n8EApGFpp5xBH0c4xlim48MIL07KddtopiaYQzfSIB6Qb/vbhhx82PXe2ox2uGucfoWOttrt56BrX\nBNEbnwv7WveDVtJvNf/03+ET9ohe3tZQ1h999FEq7yj7GHMHs2fPbnrPbnXfXNhzfqFL7rvvvtIv\nSKvV9ELj7LfffuU21CUeWAg/jofTbvRFhb3CqKGwbydutJmwf+WVV3pULCwqZwzWoSHJQ3zym9Y/\n/vGPXjH80Uh04sA1/bfzCb/LwxoiBpQZRWrd7BpZLtAU9iNL2NczQkl4QAxabeeijc17B6MdPe64\n41IoQQiIePDMOfTQQ5sOQmzFZ2P/VtvdmEFq8uTJPY5Vfctb737QLP1W80//HT5hX207KXtmAXvw\nwQfL9ZTbm2++mXyi0T271X1z/6lGEcQsPqTZanpR/0jvlltuSQ8owN/4v1t9UWGvMKoJlYPelXoD\nCfsi7CMur5Hlwp5j//GPf6ybvsJe/+0Pec8jvU8HHnhgsjFjxtTsDY2bHa9jmfINI4aTh196cKvT\nZCrsR5aw51X/66+/ngTBaaedVjccptV2LiYnOPfcc+seO3yIAYtVIkSnkX/FuTPIkbBJ3hJUrRrD\n3KzdveSSS3q8oager5mwb5Z+q/mn/w6fsI8yo8c6HxyNTzBrXnXGqDxMrd49u9m+ccy8979aF/I0\nm6WH30f4ZB5qQxz+J5980lZdtu1V2HcF8dq02aveU089NVUcenGaCfu4IWBMmxlCisFWDLLib61B\nuAp7/XcwOP/885s22hdccEGvm12rszko7EeWsK+Wex77nocCttrO3XnnnQ2nV4X4FsL9999f178a\nPRi0E2Pfarsbx+2rsG+Wfqv5p/8On7BHE1DOeVnnYWmEshA/z8Dp6pSq1Xt2q/uGn9SaPjUX9u2c\nC/DQS8dO3u5zXW+99VbX+qLCXmFUl3jVVA1JCPL4T17HNWvI582bV8ZX1uo9JQaOtwMKe/13sOH1\nfwycQrDlvZn05ETc76KLLpq2bVckKexHnrCvVe4xGJBe6Ii1bbWdi9f81faVto03QcTY8zaAbY46\n6qheaZ1++ulN/XEwhH08bFRDcWLMQH+Ffav5p/8On7BnJpkYB0IYWYSM0cmXf3iK8mx2z2513/yN\nT4zPqJ4PD9itpoePMvtZ9M5zD+DDWzEbIG/JutUXFfYKo7owxVPkB1/PDIEDDBxk2ql8BpFmDXn+\nlE+FCYhno7KGyOqLsHfwrP7bDky312hweD5QK3omFfYK+1qdG9F2hUBvtZ2L7ehRfOGFF3r5JiFe\nhHdFDyKDtAMeCvJOlWbnHnH0+HrVovez1XY3OnQ472h3EXfxXZP+CvtW80//HXyfr/oN930Gjce1\nxsDr6Nnmmw45TG9ZnRazes9udd+8Tb788svL7ag74XeI7VbTi7FU1fErTMcawr5bfVFhrzBqyEUX\nXdTjlRZzLOevrRp9ebbWl+LOPvvscl96fIhNjhtUDNSt94W5WsvzV2XMTKI40n/buXHV+0ZD3sCH\nX7Y7WEphPzJ8t1m5R0hNvk0r7VzuYxhT9NEzX/34T/RGYvx/4okn9pq6r9m5NzMEWjvtbn5foIcz\nRE6t6S4b3Q/qpd9q/um/g+vzjYzZm4IYiIoxqJs3TWPHju3hExHSW71nhzBvtm8+Q03MZpPXhQhp\na/Vc8ogDfJjQTOaqD11DKE63+qLCXmHUlMcee6xHxQjj9RTT/eU9L4Q35D00+e+AmUPyeY8x4tV4\nlVsrnXrpA5UvbiqkqTjSf1shQgaqA15z8K1cqMTMC9FL1YwIWaCnSLrXd6Pc6/XM0VsdQigfJNis\nnQuuuuqqXvG9PCzk8ACab4Pw4MNqzUQFbxFaEfYRStlquwuEPORtL+fDX8LXIqSylftBX+8T+u/g\n+3zVKCdizWu9JYqwsdyPiX1nkoH8obd6z6ZtbWXf8BPsoIMO6rE931Xoy7nQlsdkCfk15r3z3eiL\nCnuFUcsw+v39999Pr+P6G1fG69+YB3wgYtTy2R0UR/qv6LudQKvtHDHD9CKyXT71ZXUb0iImeDjb\nOsIdeGBgVp9qG4xgyx9uhir/9N/OIPwYH63nx/Xu2e3sC2zXSIu0kx4PF/jYZ599NiJ8UWFv5RQr\nqIi+Ky0L++gFJYwCcYZ4it7Q6qBa/VdEYW/lFNF/Rd+VDqVeiA899tWPB+m/Igp7K6eI/iv6rnQw\nzGoyadKk1FPPfN58zIcBjvqviL6rsBfRf0Xf1XdF/xVR2Fs5RfRf0XdF9F8Rhb2VU6ygIvquiP4r\n+q7CXkT/FdF3Rf/Vf0XfVdiL6L8i+q7ovyIKeyuniP4r+q6I/iuisLdyihVURN8V0X9F31XYi3hz\nEdF3RfRf0XcV9iL6r4i+K/qviMLeyimi/4q+K6L/iijsrZxiBRXRd0X0X9F3FfYi3lxE9F0R/Vf0\nXYW9iP4rou+K/iuisLdyiui/ou+K6L8iCnsrp1hBRfRdEf1XRGEv4s1FRN8V0X9F31XYi+i/Ivqu\n6L8iCnsrp4j+K/quiP4rorDv8sq5xBJLaB1o3lz0cX1c3xXpdv+1rRu97a/C3kqnKez1cX1c3xVR\n2GsKe4V9fyvdZ599pnWAKez1cX1c3xUZacLetm/0tb8K+2GudJ9++qnWAaaw18f1cX1XZKQJe9u+\n0df+KuytdFpW6aZNm1baTTfdpP/q4/q4wl4U9ra/Wte0vwr7Ya50n3zyidYBFuVBRctN/9XH9XGF\nvSjsbX+1bml/FfZWOk1hr4/r4/quiMJeU9gr7Ptb6T7++GOtA0xhr4/r4/quyEgT9rZ9o6/9Vdhb\n6TSFvT6uj+u7Igp7TWGvsLfSWem8uejj+rjCXhT2tr+awl5hX3z00UdaB5jCXh/Xx/VdkZEm7G37\nRl/7q7C30mkKe31cH9d3RRT2msJeYd/fSvfhhx9qHWAKe31cH9d3RUaasLftG33tr8LeSqcp7PVx\nfVzfFVHYawp7hb2VzkrnzUUf18cV9qKwt/3VFPYK++KDDz7QOsAU9vq4Pq7viow0YW/bN/raX4W9\nlU5T2Ovj+ri+K6Kw1xT2Cvv+Vrr3339f6wBT2Ovj+ri+KzLShL1t3+hrfxX2HVzp3njjjeLb3/52\n8Zvf/KbH8vPPP79Yc801i3fffberHPuRRx4pll122ZrnXW9dvvyBBx4oFl100eKdd94pHn300bpp\nKey9sWgKexGFve2vwl5h3xXCftq0acXyyy/fdcL+wQcfLBZbbLGa5/3kk0/WXJfv88orrxS33npr\n07QU9t5YNIW9iMLe9ldhr7Af8kr33nvv1bXXX389CfsXX3yxx/IQ9vRcI2yPOuqo8jqPPfbYtJzt\n7r777tTDzfItt9yyeOGFF9LyJ554olh77bWL8ePHFwsssEDx+OOPp+Vvv/12ehPAfvymh3yZZZZJ\nDxZx3JNOOin9f8stt/RIe968eWXa2267bXkOpB2/H3744STGY93ll1+ejo9tvfXW6dixLizf56mn\nniomTpzYK62XX365WHfddYvDDz+84XU3MoX98Pi4NnSmsBcZfcLetm/0tb8K+y4X9tdcc03xta99\nLYWm4CRcK73ac+fOTf+feOKJxZw5c5KIR6Qj3glvYR3LEOq//e1vy7R32mmn4sgjj0z/H3fccWm7\nu+66Kz1AcMwrr7yy3J+0Ee4bb7xxOk/S5mEgF+/571yMs5w0ZsyYUdx7771JiDcT9nla/M/58FZj\n6aWXLjbYYIN0/EbXrbD3xuKNRWEvorDXFPYK+0GrdAjmevbaa68lwfzSSy/1WD516tQkan/3u9+l\n/+nxvuKKK1KoytNPP1289dZbaflCCy1UPP/886nHndAV8oEHAMQy+9CTXT3m1VdfXQphesHZ57TT\nTktpxD5nnnlmKcLZhwcD1vFA8dBDDyXxzbmxLv+d/19NgzcAHDf2C6u3f1zDOuusk/KIa468qXfd\njfJaYT88Pq4NnSnsRUafsLftG33tr8K+w4U9IvW+++7rJexDfCNoCU+J61x//fWL2bNnp179WJZb\niG/SJf3qMZ977rkkmO+///4kos8999xi7NixKWwmhPjZZ59dHHLIIT3Okx73doU9bwYijeiZb0fY\nxzVxLTzUsH2j61bYe2PxxqKwF1HYawp7hf2wVLpXX301iVbCbfLl++67b7HDDjuk/+mhJ/acsB16\nvRG+iG6EM736iP94ALjhhhtSjzsCuVbvOIZwp6eenvCtttqqFPrk4fTp09M2pM266gMI4jlmruGY\nrIsHhFrCngeGSCPfrhVhz//01HN+Sy21VLHnnnuW51bvuhX23li8sSjsRRT2msJeYT9olQ4hXc8Q\nsMSIL7jggsXNN99cPPPMM6m3nuu56KKL0jYMlkVUI3B5EFhhhRWSsCcunu0uvfTSFId+6qmnpt8M\nco0ZZRC+tY57zjnnpG3pmeccEMpf/OIX08w1rL/tttvS+uuuuy4dk553zpFBrIS8sI6HEc6JfddY\nY410rPy4kcaNN96Y9vvZz35WbpefS75PiP/4Px5O7rjjjpQWDzaNrrtRXivsh8fHtaEzhb3I6BP2\ntn0DZ4Qdv/nmmx3f/irsO7zSESu+xRZb9AgrOeaYY0oBTBw54TexbrXVVkuCmnWE0cRyhDkDX0Ms\nRyhPrWOGOGdQK7+PPvrotH3u0Ij+SJse/dtvv718GNlvv/3KdSuvvHIPYZ8f9/TTT+9xXfWEfexT\n73+223///YtFFlkknWO961bYd+6NBZ/Dj/D3gWqE481UPT8fSGtWp3K74IIL0oD4RtsQfsebL944\ntZO2wl5E/63V/tIJR8dYNUyVmeNCMwxE2zVS21Q0BvlFHpKGwt7KWbfSIYRbMSol4ST0QtdaT4w5\nVl1OmAz74cCtHqudc6K3vZ3zaSeN/li7162wH14fp6FF2HODGSgfoCH/wQ9+kBrhgfavqvFg0sqx\nCJnjYZObUKPtWM/brIG8DoW9yOgT9lH/6W3m7f6dd96Zwnd5A0/bQscgYbED1XaNxDaVY5B3TC5C\nPnZ6+6uw7xJhrw2uKew7W9jTIMe3CXiDxdiSZutIMxpvGvWf/vSnxWGHHZZuGNtss03ZqDN9a/xm\n3U9+8pNit912S+mNGTOmeOyxx2qeE+Fe8R0GxpzwxinS5OaZn9Ozzz6blu+yyy7lgG+ulZ6mGMOy\n+OKLp3EqbMf3IPbYY4+ydymugwfVI444ovQL3qa1enNS2IuMbmHPW+1qRxpTTscb+VbbLt7QN2pD\n11prrfT9Gto2xPAmm2xSTJo0Ke1PuDDH6JY2lXxjHF9EAHAe9e451WsnXxT2o1DYxyBPbXhNYT+8\nPh7Cnsa61jry4Pjjj0+zIW200UbpBkUvSqN1hJItt9xyZcNMz1Qsp1HnRkb6+e9Ib7vttku9WnzX\nIdLLz+mee+5J2xH2FTccbkKkwaxUcU70inHD41sLpMF4EMajMD0tNxy2u/jii9N2PEywHW/l8nNi\nQHr8T1gZ+3Oe119/fdofP+1EH1fYi8K+c9pf3qDHZBz0WNPGIKRZtv3226dtWm27GMvWrA1l3xNO\nOCE9BPAbsctxQ7DHvt3QpvIbUT9r1qw0lq/Z/SiunVDp4Wh/FfYKe01h39HCnrEY0XPDb77rwLaU\nT6N1kSZfWc7Fed6oV3/HPhHGFekRr9/onDgeNxDSmDJlSrpZci2MGWCgN2VIGoSIcS6kTw/Stdde\nm/anF40B8ZFGfk75/6TN+Vx22WXpphEzYinsRfTfZsKedqQaY0/sOO0P27TadrXShkZbXut3vm83\ntKn5MVq559S6jynsFfZaH43K3KrQUdh3h7BnClNe4+Y3KHpzKJ9G66L3JF7TRu9Jo5tSdV2kxwxQ\n1ZvQ4Ycf3uP8Y7+YVapqnBPpcYPgOvHTgw8+uNcA8kY3IfbZfffdy+3XW2+91DOlsBfRf5sJe9oe\nZtfjNzPtcX0nn3xyuU2rbVezNjS+LRPrGnWsdEObmh+j0T2neu0K+1Eq7HnqCzvvvPNSjFi+rC92\n9913F9/97neTw/Y3rUbGU3O85mq2bSvXlp93O2mHxWw8VNh2r11h3xrcFPrj4/WMBjFmxamu48vH\n48aNK3/z8PbNb34zlU+jdfhQ3MiWXHLJMr6SV755/cDv8Bl8jX3Yn/CdPD1e41bPiVewue9GGqwj\nBIj0MXp76EViylXS45y4TmaW4hUwr7XZhjTC5zmP/Jzif244jz/+eLpxEOfJcm5knejjCntR2HdO\n+xttD+1QLKO3mmskXCbatVbarmZtaH7vrvU7tu2WNjXPu2b3nHq6RWE/CoU9s8MQw1VL2LRrpMFH\nmQZT1GPEl9WqoFVr9dry86bhaCXt/BhUrquuuipVtHavRWHf+vkwoHTmzJkDLuzxEeIkiZkkjpEe\nJRrS6FliEBZlS68OjTcDYhutw4fiNWzEXlKmvL7lf3yF9LlhrL766snXOA/W0SuDT5EefsX/+fnG\ncYnJ5Fgbb7xxmUZ8V4E4T/Y75ZRT0m8GWPHWgPT4kjPLYzAb58GgXdbxO/f//CbEVLdsM3fu3LQd\nA9EU9iKjw3/70/7WEva0KbQhdHwghFttuyIUplYbWhXuzX53Q5ua512je069a1PYjzJhj+jAfvnL\nX6bzxMFwHpybJ2KWbb755inuK7YNmzhxYnHhhReWv3E45qRFuDBohCdblvMxqDwtHJjKQaVhHdvw\nhIsI4tj85sn3uOOO63VMKlaMWueplTQaHafWtZ1//vnlqPUf/ehH6bzZjhHlcd4xHRX/U0moPFGW\nfBArjokR40bDFKPWOY96+UferLnmmmlwC+fA6z6WK+zbOx9s4YUXTh8A+/zzz1vy8UaWh83kxkAv\n1p9xxhk9vpsQPtNoHT4UA6z4zRebaZzpjd9nn33KfVZaaaXSj2mYSSP8if+pG7XOOT5+FpbXhfzV\nMT7JTYfl+DL1huVsQ52I7TgntqUe5Oee/89bL14Vxz6rrrpqWc+amcJepPuFfV/bX+6TIU7zdiFE\nOu1Rq20XorZeG1ptd5v97oY2tZp3rd5zhqv9Vdh3iLBHjPLUxwAOXh9xzghrBM+GG26YnAoxnjtK\n/nqK3zvssENy5HiFxXJeM0VavG6K0eSkxfZM/8e+Rx11VNqOnk0qCukyYj4/XnzVla/f8gSNcI6K\n1+g4+bXFiHIGtbDdrrvumraL12Zx3rmwpxeX/elN5fUb+9Ozn58br9CislNZ6uVfNGKcH+sZDKOw\nb/98iCOM/+eff/70kInf9FXYt2L0jtAr0u66ekZPD5Yvw++iTtHg9+eceLVLTCb5Umu/+D++Gt3f\nc1fYi4wOYT/Y7W+rbVdf2qGR0qa2e89R2I9CYR9zzOKQCPZ4BcU6nIenQsRr7ighphkJHtvwuikX\nxfS8x2smKgSxwuyD4EUwh6heZ511yoE0bEda1Sf76nkhruPptNFx8mtjTvDohaUSMeds/hRdS9iT\nNufDgwZCnOuuVtpW8y96ZKvXprBv/3x487LKKqv06GGZMGFCKrvBEPZDYTy88sBKneim81bYi4we\nYT9S29+RbAr7USTsEZ9YvOpBGDM6/Ne//nWPdYgNhHQswxC9vEJC+PJqjG0QuDFfK+KEdbVCHEhr\nzpw5ZagB2zOQht5twm2iJz4/Hq/JDj300PJ3iO9mx8mvjfM76KCDer1uI438vPP/2Sc+GIStu+66\n6QEhP7dW8490eQBhWb4/ZVHr/LXalkOejh8/vsd64kCrPt4NxsMmb5i66ZxbNX1c02x/teFrfxX2\no1TY09uNgCZWPl+HGK0KewwRTpw6Pe4nnXRSjw8xIIpJa9lll0093BgimR5zBDDCnf34UhrHYyBK\nxL4Tq1Y9Fmkh/ON3PBA0O05+bWeddVYKqyGUh20itr+RsOeNBDFypEPsPMtz4V4V9o3yj3TjeIqe\ngbmx4EeTJ08u5ptvvh7beGNR2GuaZvurKexHlbCP6ZuYEQbxSbgJc2ZzznwRjR5EeskRw4SZxPZh\nhJSwH/HlxKyzDLH8/e9/PwnsmA2EQbbEhCH++R0fYWD2D37Ty43YRZyTFjPeVI9FKAvbEufOuYwd\nOzaNOm92nPzaWP6tb30rXRdTd6222mppHeEy+XkTUhT/E//PNrxhYL/ll18+9frn58Zy0iXNOM9a\n+Zenm+9vKE7758ODYLWniHhPXgfnDVnVj9o1wq+IrexvOqPdDMURGTmhOEPV/mrd1/4q7DtE2COq\n6WWOEecI7XzUNQ5Qz2H22muvNPMLafA7pvkL8Rpz1cZocgaxxr4xvR896Pxm9hn2rSekiMHPG5IQ\n9o2Ok18b2+Sj1vfee+/yQSI/7/x/Hl4Iv8lfM9Ko1RP2/K6Xf9W8Udj37XyqsZ3M0MADVD5Dw0Dc\nWGLmBR7GeMPTKY10PT9qx6ZPn16+Oasa9Y+HbOp1f46hsBcZecJ+qNpf21eFvcK+H8I+F6j5//R2\nD4RTEY6Ck4f47481Oq9Gx8mvDbHebi8sPbfYQJzncIuebhf2YYsvvnjdOZX7e2OJD38wyDv3nU4w\nBoXXevPTzrXxQEs9qLU+3oBhPHwr7EXUDkPZ/tq+KuwV9n0U9ghgbfhNYd/e+fCFwJh9YbB8PP82\nAdOkEgIW3ybYdNNN09gLtqNh5rPhW221VfHlL385zXwUabCOeYl32WWXtN/KK6+cbhr19qt3DIww\nM7bDeAPF2yreIpAOafA/25FO/puxJnEtW2yxRQop23nnnctvO/D2qXrt2223XfpeA28seKvVjT6u\nsBeFffe2v62Y7Wtntr8Ke4W9prBvmYjfHAofZ6wJop6xGQx8Jh/4SiAN+wYbbJBCr+KLgqyjsT/2\n2GPTIO1II9YRi8pMSny7od5+8UXBWseILx8SSkZvDzenuPHceeedZagQx8x/x1duI01e/3Lz4UGF\ncR+XXHJJSj+/bsLJYqrY+AYEPU8Ke5HRrR2Gsv1txWxfFfZWzhqVLmaR0YbXFPad5+P5uAnGdkRD\nzzrEOz07iH8+bMb/9NRU04h1pJHvx+xI1f0aHaO6jp6npZZaKoWTkQ43Gv6PY8Zv9ovtWMeNiPmn\n82urnvOMGTPSOsLaGPDNTY5l3ebjCntR2I9sjWH72pntr8JeYa8p7DvSxxlPQQPMjYEpTA888MAe\n62iQQ9jzypVltW48+U0h9mPGpOp+jY7BDeSQQw6pmW6jGw9p7rTTTg2vLV/OjW3ttdfuFUvL9yro\neVLYi9j2dpKwt31V2Fs5h6jS4dg4f+68hDUw3WQ8FfenMudPyX2xc889t+bTNMaTNK/UNttss34d\nQ2E/coQ9DT+vWPN13DRC2Nfzx7i5MJ1qvh+vc6v7NToG63h1HOtuv/328ubCa2RuUHFjyNexH74c\ndY7X1cR1EvdZq0eJuNmYGYtXy8SXXn755WkZ56ywF7Ht7SRhb/uqsLdyViodDjkYFnPHX3XVVamC\nYbxuIi/425+0qRxULCpaX/anEeAhg8pXa33E4mG80husPMpNYd95Po4P0zjPnj07fZiMfGCGHJbz\nQZavfvWraQam+P5BLX/E79mPnh38jv2q302I/RodI9ZRn/jN4DVeHbMvPso6pnfluw3LLLNMuY4Y\nzjxNepdYxw2H8+ADc7w2jvM9+OCDy31jGTew5ZZbrpg4cWJX+bjCXhT2I1dj2L52bvursB/Bwh5R\nVBXPzNvK6zDE+eqrr16OWKfy8BT7ne98pxyxztNv7MeUWjFinafuqBykM27cuLKikE7+m4oaX7zk\n2FTMX/ziF+WIdURb9dwZsX7EEUek0erM0a+wV9jzm5tH5Ad+yM0gvl5Mz1A9Yc+24YP8z82g3n71\njoGdeOKJvb7hwL7cGPDTWL7iiiv2uHnwteVYh89TZ9iHehQfUovr5UY4derUXtfBl6C5CSrsRWx7\nO0nY274q7K2clUqHAw6GMSgEUcTXaHli5Qma0eZUpvPOOy9VOvIFEX700UeXs47wlVdGs6+//vpp\n/9iPdVQgetMR/3wxlpAZKjGVhf85bv47Pn4VafKkjbjny7BUoosuuiiln583YRfsQ+VkZDvb8XAy\nWPkUprDvDh+ncaZHp9Xteb2K3+GPrfpRo2M0WsfMCvF9hlr74du1lg+2bw+XjyvsRWE/cjWG7Wvn\ntr8K+xEs7Ksx9vHFOioAwh6RTw862xN7j1iPJ2QqF+uZI7a6jp59nsSpzDH1VAj7/Df7xXasY3op\nHioidrpWQ0DsfTxQUIl5iOALcgp7byx9MR5K8SEebjulgR8NNxaFvSjsR377a/uqsLdy1qh0iOXB\nMKaS+sY3vpHmAMd5GKCCmI/1DEhhPdvxm0EoBxxwQI/9qbCIeF6RTZo0qVwX4p3KTDrxf6Qbv0lz\nxx13rHluiPcnn3yyx3IeANZaa61eDyPrrrtuEvqDlVeYwr77fLwV4+GQN0TDeQ6dYgp7kdEn7G1f\nR1/7q7AfwcI+ZhSptR4BztfaQpAj3pmFpvpgEMKe0JxYx1N6iPd4Yg/hna9jv3hNx7qHH344xcoR\nM13r3GIgDjFxhO4Qr8+DCcsI8VHYe2PRFPYitr22v7a/CvtRK+yrveK5sM972vkgRIw8Z1/mlI34\n9liH4OY3I9YJzWHfEOMIcD4PjZCPdRGbH2nSe886BD0PDXxCmrCcOKdf//rX5b55Lz4j1vfYYw+F\nvTcWTWEvYttr+2v7q7DvXGGPcB0M4xUZwh6xXWs9c8HSY09PeywjdCYfsY6gj3UnnHBCj/AYBDj7\nIsKZJiqWr7DCCuU69jvjjDPKdYh5HgTYh0G0LGMO2Tjf733ve+mT0tVzPfvss9NDxmDlFaaw7z4f\n1zrbxxX2orC3/dWGvv1V2Fvpehg96/TKt7uOeWOxevsxSLfW8tEqehT2msJeRGFv+2v7q7AfYcKe\n3mtt+E1hr4/r4/quyEgT9rZ9o6/9Vdhb6TSFvT6uj+u7Igp7TWGvsLfSWem8uejj+rjCXhT2tr+a\nwl5hnwaZasNvCnt9XB/Xd0VGmrC37Rt97a/C3kqnKez1cX1c3xVR2GsKe4V9fysd87hrw28Ke31c\nH9d3RUaasLftG33tr8LeSqcp7PVxfVzfFVHYawp7hb2VzkrnzUUf18cV9qKwt/3VFPYK++L555/X\nOsAU9vq4Pq7viow0YW/bN/raX4W9lU5T2Ovj+ri+K6Kw1xT2Cvv+VrrnnntO6wBT2Ovj+ri+KzLS\nhL1t3+hrfxX2VjpNYa+P6+P6rojCXlPYK+ytdFY6by76uD6usBeFve2vprBX2BfPPvus1gGmsNfH\n9XF9V2SkCXvbvtHX/irsrXSawl4f18f1XRGFvaawV9j3t9LNmzdP6wBT2Ovj+ri+KzLShL1t3+hr\nfxX2VjpNYa+P6+P6rojCXlPYK+z7W+nmzp2rdYAp7PVxfVzfFRlpwt62b/S1vwp7K52msNfH9XF9\nV0RhrynsFfb9rXRaZ5nCXh/Xx/VdkZEi7LXR1/4q7K10msJeH9fH9V0RhX3HWZSD7a/Cvisr5+TJ\nk3sVuja8pv/q4/q4visKe9vf4bEoB9tfhb3CXlPY6+Oawl5E/1XYK+wV9sPDtGnTtA4yhb0+ro/r\nu6Kwt/0dPotysP1V2HtzEW8uIvquiP5rOZhnCnsR/VdE3xX9VywH80xhL6L/ir6r74r+K5aDwl6n\nENF/Rd8V0X8tB8tBYa9TiBVURN8V0X8tB/NMYS+i/4rou6L/6r+Wg3mmsBfRf0X0XdF/xXJQ2OsU\nIvqv6Lsi+q9YDgp7nUKsoCL6roj+azmYZwp7ERs1EX1XRP+1HMwzhb2I/iui74r+K5aDwl6nENF/\nRd8V0X/FclDY6xRiBRXRd0X0X8vBPFPYi9ioiei7Ivqv5WCeKexF9F8RfVf0X7EcFPY6hYj+K/qu\niP4rloPCXqcQK6iIviui/1oOorAXsVET0XdF9F/LwTxT2IvovyL6rui/Yjko7HUKEf1X9F0R/Vcs\nB4W9TiFWUBF9V0T/tRxEYS9ioyai74rov5aDeaawF9F/RfRd0X/FclDY6xQi+q/ouyL6r1gOCnud\nQqygIvquiP5rOYjCXsRGTUTfFdF/LQfzTGEvov+K6Lui/4rloLDXKURGm/8uscQSWgeaviui/1oO\n5pnCXkT/Vdgr7G17xbZXLAeFvU4hMlqF/WeffaZ1gCnsRfRfy8E8U9iL6L/9Evaffvqp1gGmsBfR\nfy0H80xhL6L/KuxHkLCfNm1aaTfddJO+K6L/Wg7mmcJeRP9tTdh/8sknWgdYlAdiPjd9V0T/tRzM\nM4W9iP6rsFfY2/aKba9YDgp7nUJktAj7jz/+WOsAU9iL6L+Wg3mmsBfRfxX2CnvbXrHtFctBYa9T\niCjsFdUKexHbXrEcFPY6hVhBu17Yf/TRR1oHmMJeRP+1HMwzhb2I/quwV9jb9optr1gOCnudQmS0\nC/sPP/xQ6wBT2Ivov5aDeaawF9F/FfYKe9tese0Vy0Fhr1OIKOwV1Qp7EdtesRwU9jqFWEG7Xth/\n8MEHWgeYwl5E/7UczDOFvYj+q7BX2Nv2im2vWA4Ke51CZLQL+/fff1/rAFPYi+i/loN5prAX0X8V\n9gp7216x7RXLQWGvU4go7BXVCnsR216xHBT2OoVYQbte2L/33ntaB5jCXkT/tRzMM4W9iP6rsFfY\n2/aKba9YDgp7nUJktAv7d999V+sAU9iL6L+Wg3mmsBfRfxX2CnvbXrHtFctBYa9TiCjsFdUKexHb\nXrEcFPY6hVhBu17Yv/POO3Xt1VdfLRZbbLHyesO23HLL4rnnnmu4b1/tvvvuKxZddNHirbfeGvC0\nH3zwwWKZZZYp3n777abbXnDBBcWLL77Y8rm2k3YtU9iL6L+Wg3mmsBfRfwdV2C+00ELFnXfeWcye\nPbt46qmnivvvv79Yf/31i7Fjx/ZZxDay3/zmN0nQDsZDw2OPPZYeVJqd9xtvvFF88YtfTOfS6rmS\nL62krbAX0X8tB/NMYS+i/w6KsP/d735X1377298WiyyySPHyyy/3WD5jxozUO/3mm2+m3wh/eq7J\nky222KJ49tlny21vvPHGYqmllkrrxo0bV66rt88TTzxR7LHHHqkXfK+99iouvvjiMq3bbrstpcFx\nGx0zt0svvbRYYIEFkm211VbFGmuskdJudA677LJLWsZDDW8m6L1nf5Ytvvjixa233trrXOmx/8EP\nfpD+R9wfccQRpZ8cffTR5THrmcJeRP+1HMwzhb2I/juowh5xe+2116YeaUJPLrvssrRs++23T9vM\nmTMn5cXxxx9fPPnkk8W2226bRD+93gjfWPfII48Uyy23XBLmjfbhGCGQTz/99LRPiOIdd9yx2Hff\nfRvun5//vffem7Y799xzi7vuuiuJ+BD2jdJA8C+44ILFrFmz0sME211yySVpu9122618qMnPNRf2\nV155Zdr/oYceKm644Ya0PyJdYS+i/1oO5pnCXsRGbdCEPb3L9eyVV15JYrUaY7///vunXny2mTJl\nShL69HY///zz6QGAbRC1CPOll146iWC2RejTA99onwceeCAdk30I/2E5gvqll15KveaI6Ub75+fP\n8UPI8xuhHOfTKI3XXnstval44YUX0jnzYMP+XPNFF11UppGfa/4/aXOuPAQRrkMI0+uvv94wrxX2\nIvqv5WCeKexF9N9BFfYI3GeeeSb9vvnmm9N1n3zyyeU255xzTi/hHz3UZ555ZrHzzjv3SrfRPrlA\nRpATz8/2iGt63BHIjfavCvvDDz+8/B296qTdKI24bkQ/xzv44IN7bMPDQiNhzz677757uf16662X\nHk4U9iL6b7fD/SDCEkVhb+UUK2gXCnsEblWU33HHHaV4JlyGEBYMUXvdddelffJQGralB57e/pNO\nOqnuPrlAZh9i5H/0ox8VP/3pT4tTTjml6TGrwn6jjTYqf99zzz1l2o3SiOumx57ed8JqCOVhG9Jo\n1mNPDz09/YQyIc5ZPmnSJIW9iP7b9cycOTOVAW2rKOytnGIF7TBhj+iuZ4SeIHDnzZtXLkMEr7DC\nCsWSSy6ZQlYQ+OQFITbMooP45vfTTz9d3H333en/q6++OqVF7/3qq69exq3X2ifEN8fheITJfPOb\n30yz1NDrzbJGx8zPP94wXH/99Wnqyo033jgdn7QbpUH4DMe8/PLL0/IYQEw+/OQnP0nr+J2fK6E8\n8f8xxxyTtpk7d27ajvyi179RXrcr7DmOba/Y9spQM2bMmFQGp556qpmhsLdyihW024U9Ru801z91\n6tT0m7+RJwjwK664otw2X4fYffjhhxvug0CmRzyEPbb33nunWWsiPKfZMXPj5pOH0YSwb5QGx2GQ\nb1wj5x3bMXiXbZk6Mz/X/H8eRgi/iX1WXXXVXnnYV2HPW4W4sdr2im2vDCWMcYoy6Gvngr6rsBfx\n5jKIwh4ROxBG+AphLDT29da1s09/j5kbDyj02LebBvvF/0x7Sc9+O+dHzz/WyraNhP3nn3+eXn8v\nvPDCPR5SbHvFtleGCtoh4urJ/wkTJpghCnsrp1hBR7Kw1/pntYT9eeedVxx66KHF/PPPX/ocM/ko\n7MW2V4YSOj5C1NMeIfJFYW/lFCuowl5rQdgzsJeQobx3ngHE559/fr/9z7ZXbHulVZgBh975yHdE\nPctEYW/lFCtohwr7mBVGG16jLAi1+T//5//0EPQbbrhh+tBWLf/TNE0bKmNsj3H1CnuFvVhBR7iw\nJ3adaR0V5/0X9l/+8pd73Ejnm2++Yo899kjTbCrsNU0baqOzgR57Bs2Kwl5hL1bQLhD2iMa+2j77\n7JPy4fvf/34S9/1JayAt5ppnsGtf05g+fXqarrLWOgbcLrvsssXmm2/er2PkFuVBXD299F/4whd6\n3GDHjx9f9twbiiO2vSL6rsJeRP8dMGHPjDFMA8kUkcz80imiHuMLszxs9FV0c21MackUlbXW33rr\nraUPMPXnQAr7GDhLvu600069ZsJZZZVVFPZi2yui7yrsRfTfgRP2fKAq5n+/5ZZbimuvvbb47ne/\nm5Ztttlm6curIbLXWGONYuutty4WWGCBHkKYdauttlrxq1/9Ku238sorp2X19qt3DOzCCy9M22Fb\nbrll+pAUwp50ttpqq1Lkk07+mzTjWpgrnw9U8SGtmHefgWLVa99+++2Lo446Kr2xYI79wRD2+XSX\nTHUZc9c73aXY9uq/ou8q7EX035rCnq/H9sX4UBKi/tJLLy1uuOGGlB98cZUPNW2wwQbFt771rfLr\nrKxDTB977LFpzvhII9YRZsIHn3bYYYe6+8VXZGsdI74iO2XKlNSbjvhH2BMedOedd5ahQhwz//3Q\nQw/1SJPwGsQ9DyoLLrhgcckll6T08+tG6LMPDwx8QZft6Nnvaz6GtfKBKuJc8xkqbHvFtldEYW/l\nFNF/+y3sX3rppSSsiUM/5ZRTyh5y1vExJwaCIv4R6PxPT3g1jVhHGvl+CNjqfo2OUV3HsqWWWiqJ\n97vvvruHsM9/s19sxzq+ikvveH5t1XOeMWNG+UDBF2Z5iCAufiiEfcBg275+IMa2V2x7RRT2Vk6R\nESrsEcN9sRdeeCEJ3Dlz5hSnnnpqceCBB/ZYh+BFYCOkCWlhWTWNENkMRs33u/LKK3vt1+gYJ598\ncnHIIYfUTLd6jPw3aRLH3uja8uU8AKy99tq9QmLWW2+9JPT7mpdYO8Letldse0X0XYW9iP47KMIe\nYU0IS74OUR7Cnl7xENZVYc92TJmZ70e4THW/RsdgHaE5se72228vxTthOjwAhPDO17Ef4TcIdtYR\nDkTcPG8JuDbCbvLz5U0C5X711Ven0B3i9S+//PK0jHNW2IvY9ooo7EWsoF0t7K+//vqUH7NmzUrL\nJ0+eXMaeV3vMq8Ke/U477bQ0sw77Idb5v7pfo2PEOgQ3v8eOHZtCc9g3xDgCfPbs2UnIxzpEfp4m\nvfesIwSH87j44otTWE6c76RJk8p981785ZZbrthzzz0V9iK2vSIKexEr6PAJe4RpX4z4coQ9Ypnf\nhLVEvhD7jthm+V133ZV63ukxr6aBeGdbziP2Y0Bqvf3qHQM78cQTe4THIMDZFxG+1157lctXXHHF\nch37nXXWWeU6xDwPAuzD2wGWERYU18uDxtSpU3tdxznnnFN89atf7XNeYgp7Ef1X9F2FvYj+OyzC\nvp7Yp8e81e0R78sss0wS2YS/9PcYjdYxGw9Wax299bWOT3oDmT8KexH9V/Rdhb2IN5dBE/b0Tg+X\nMTXld77znRR6M5zn0QmmsBfRf0XfVdiL6L9dK+zpQUe8jnZRr7AX0X9F31XYi+i//Rb2hMFow28K\nexH9V/Rdhb2I/quwV9jb9optr4jC3soporBXVCvsRWx7RRT2Vk6xgna9sGeedm34TWEvov+Kvquw\nF9F/FfYKe9tese0VUdhbOUVGu7Bnqklt+E1hL6L/ir6rsBfRfxX2CnvbXrHtFVHYWzlFFPaKaoW9\niG2viMLeyilW0K4X9i+++KLWAaawF9F/Rd9V2Ivovwp7hb1tr9j2iijsrZwio13Yv/DCC1oHmMJe\nRP8VfVdhL6L/KuwV9ra9YtsrorC3cooo7BXVCnsR214Rhb2VU6ygXS/sn3/+ea0DTGEvov+Kvquw\nF9F/FfYKe9tese0VUdhbOUVGu7B/7rnntA4whb2I/iv6rsJeRP9V2CvsbXvFtldEYW/lFFHYK6oV\n9iK2vSIKeyunWEG7Xtg/++yzWgeYwl5E/xV9V2Evov8q7BX2tr1i2yuisLdyiox2YT9v3jytA0xh\nL6L/ir6rsBfRfxX2CnvbXrHtFVHYWzlFRruwnzt3rtYBprAX0X9F31XYi+i/CnuFvW2v2PaKKOyt\nnCKjXdhrnWUKexH9V/Rdhb2I/jsqhX2UhcLetldse0UU9lZOEf33/2fy5Mm9hGWnW5RFt513q6bv\niui/ou8q7EX0X4W9wt62V2x7RRT2Vk6R0ei/06ZN6zqLsujGc29mCnsR/Vf0XYW9iP5rWYj5Jfqv\niMLeyimi/1oW5peI/iuisLdyihVULAvzS0T/FVHYi3hzEcvC/BLRf0XfVdiL6L+WhZhfov+KKOyt\nnCL6r2VhfonovyIKeyunWEHFsjC/RPRfEYW9iDcXsSzMLxH9V/Rdhb2I/mtZiPkl+q+Iwt7KKaL/\nWhbml4j+K6Kwt3KKFVQsC/NLRP8VUdiLWEHFsjC/RPRf0XcV9iL6r2VhRphfov+KKOytnCL6r2Vh\nfonovyIKeyunWEHFsjC/RPRfEYW9lVOsoGJZmF8i+q/ouwp7Ef3XsrAszC/Rf0UU9lZOEf3XsjC/\nRPRfEYW9lVOsoGaEZWF+iei/Igp7K6dYQcWyML9E9F/RdxX2IvqvWBbml+i/Igp7K6eI/mtZmF8i\n+q+Iwt7KKVZQ/deyML9E9F8Rhb2VU6ygYlmYXyL6r+i7CnsR/VcsC/NL9F8Rhb2VU0T/tSzML/NL\n9F8Rhb2VU0T/tSzMLxH9V0Rhb+UUK6hYFuaXiP4r+q7CXkT/FcvC/BL914wQfVdhL6L/WhZifon+\nK6Kwt3KK6L+Whfklov+KKOytnGIFFcvC/BLRf0XfVdiLeHMRy8L8Ev1X/xV9V2Evov9aFmJ+if4r\norC3corov5aF+SWi/4oo7K2cYgUVy8L8EtF/Rd9V2It4cxHLwvwS0X9F31XYi+i/loWYX6L/iijs\nrZwi+q9lYX6J6L8iCnsrp1hBxbIwv0T0XxGFvYg3F7EszC8R/Vf0XYW9iP5rWYj5JfqviMLeyimi\n/1oW5peI/iuisLdyihVULAvzS0T/FVHYi3hzEcvC/BLRf0XfVdiL6L+WhZhfov+KKOytnCL6r2Vh\nfonovyIKeyunWEHFsjC/RPRfEYW9iDcXsSzMLxH9V/Rdhb2I/mtZiPkl+q+Iwt7KKaL/Whbml4j+\nK6Kwt3KKFVQGm5kzZ5Z538wmTJhghum7ov+KKOytnCL6b6cy//zztyTs33jjDTNL3xX9V0Rhb+UU\n0X87lUMPPTTl/SqrrNJr3UILLWRvvb4r+q+Iwt7KKaL/dgOff/55mf+vv/56ufzee++1t17fFf1X\nRGFv5RTRf7sJeuTJ//Hjx5fL6MFn2ZgxY8wgfVf0XxGFvZVTRP/tBuiRjzL47LPPUs99/L7vvvvM\nIH1X9F8Rhb2VU0T/7RbomacMJk+enHru+X/hhRc2Y/Rd0X9FFPZWThH9t5ugZ54ymG+++cryYDpM\n0XdF/xVR2Fs5RfTfLoMe+igLpsEUfVf0XxGFvZVTRP/tQvIPVjENpnS/7y6xxBLaCDfbXrHtVdiL\nKI6kJvHBKqbBFIW9prAXUdh7cxEZdf6rwBh5omckCXtmO9JGlinsRd2gsBdR2CvsFfajUNh/+umn\n2ggzhb2oGxT2Igp7e0btzVTYawp7tYOoGxT2IvqvAmr0iB6FvdYNPs63J8JuuukmtYOoGxT2Ivqv\nAmpkip5p06aV1q7oGUnC/pNPPtFGmEXZ4te5qR1E3aCwF9F/FVCKHoW9prAXUdgrjET0XwWUwr6T\n/fLjjz/WRpgp7EXdoLAXUdgroBQ9CntNH1c7iLpBYS+i/yqgFPYKe01hL6KwVxiJKOw1hf2w+OVH\nH32kjTBT2Iu6QWEvorBXQCl6FPYDam+//XbxP//zP2VehG211VbFSy+9NGDHefzxx4tll122+OCD\nD4bEhx5++OHie9/7XvH+++8r7EUU9gojEYW9prDvDL/88MMPB83eeuutYqGFFiruvffeYt68ecXc\nuXOTKN5www2LTTbZJAnjgTgOaS622GIDll4ze/XVV4vbbrttSI7VF1PYi7pBYS+isO9iAaUNrehR\n2Lcu7L/97W8Xr7/+eo/lF1xwQephf++994qnnnqqGD9+fCnKn3zyyfI36zbddNPi0EMPTXm50kor\nFffdd1+v4zz22GM9hD0PEvSos8+4cePS24HY9pJLLin++7//O6373//93+Kuu+5KyznWOuusU0yY\nMCGtv/766+se++mnny722muvls5x1qxZKb0vfvGLxSmnnFJsscUWvfJDYS+isLdyihVUYa8p7Dte\n2NNjT/4+8sgjqWf9yiuvTMt22mmnmr3t+W9CbMjD7bbbLu2/7bbbFmuttVavnvlc2D/33HNpn5NP\nPjm9IUCo8xDx7rvvJsHNOsQ26yZOnFg+YMSx2B4BzsNBvWO3eo4PPfRQWnfuuecWDzzwQHrYQOS/\n/PLLCnsRhb3CXqygCntNYT+wfklc+mDZm2++mQRwNcb+oIMOSr3WbPPoo4+mbRDX1d8I9gUWWCD1\nuLMuBHxsG5bvg4jmweE3v/lN8corrySxzTGfeOKJ1NNOWbMPx7/ssstKYV/rWPWO3eo5Tpkypdh4\n443L8+TBIt92sExhL+oGhb2Iwr6LBZQ2tKJHYd+6sCcU5/nnn0+/77jjjpQnp59+ek1RXv3N/+z/\nzjvv1Ny2VhrTp0/v9SCB3X777SmdyZMn91i+5pprlsfigYBzjjTrHbvVc9x///2Lo446qjzP6kOA\nwl5EYa+wFyuowl5T2A+YXxIyMlj2xhtvJNFL73kso0edfLnnnnvS7wcffLBYdNFFkzDmNyErCGNC\nZ+htjzAa1vE71uXHyZefddZZxfLLL5/Sw373u9+l8uUcpk2bVnzta18r7r///rScY0X6tY5V79jV\n/+ttx7lsvfXW5XlyrQj7PD8GwxT2om5Q2Iso7LtYQGlDK3oU9n0X9gjeFVdcsVh66aWTuCZEhny6\n7rrrihdffDGJcnrRqwK6VWEfsfGXX355mm6TtwP8JvYeoc35cF6c0+qrr5566QnLaXas/DcCvZbI\nr25H7z3Hvvjii9Og4MUXX1xhL6KwVxiJFVRhrynsR4awx0LwzpgxI4WsHHDAAWV+rbLKKj2EfaNe\n9FzY58tJN9JjNpprrrkmLUdcI+RjHcdl/ezZs1vqsa/Vu99svxtuuKE8D2boIT944FDYiyjsFfZi\nBR1Fwh5RVGvgIeLghRde6DjBfNFFF6XZPhT23SXsEdadYMwNjw1UesTKE8uOwK6uw08R14N9TTxQ\nHHnkkeU58Js6TYjQYB5XYS/qBoW9iMK+wwQU4QH0Lt59991pNo85c+akmOANNtggzbQx2OKgHWNa\nQ3okEUydck6DKXoU9lorxpSa8RaCaTDztxQKexGFvcJerKCjTNjz2r7ai3n++ecXyyyzTIpP5jfC\nn8GH5NOWW26ZevNjW+KMielFdO+zzz7Fz3/+8/RAQFwzQiMeDpiLO/9dL016HpnlI8rl2GOPTfvs\nuuuu6TcPIsRJx/E5ztprr50+OMR5cJxG53vzzTen2GvWMegw1t1yyy099uErpgr7gfFLylQbPCO2\nn7dZM2fOTLPiDMUxFfaiblDYiyjsO0xAvfbaa0koM6iQnnoMoc6yHXbYIW3zzDPPpPw58cQTU5ww\nAhrRTw86M3+wjq988kEd9iN2OWYDISSA/0kn/90ozauvvjrNKkIc8Y033pi2Q3Tz9U6W8/Ehtotr\n4LhsQxonnXRSOad4rbSJf451CCAGUW611VZlGrF87Nix6YEnP85Qix6FvdbJprAXdYPCXkRh34HC\nvlaM/YEHHph68dlm6tSpSbAzTzgDFJmtg20YnHjmmWcmIU+POtsyj3f09PNFzFzY578bpck6et6v\nuOKK9PEfQoQQ2DFfecQ058Ke7el5b+V84/zY9qmnniouvfTSXtfx29/+NqV56623KuwV9n0y6k9f\nHwwV9iIKe4WRiMK+T8Iesfzss8+m37fddlvKi1NPPbXchnm5a32MB9GLID7kkEPKbXPx3kjYN0oT\nMTRx4sRy2frrr5963uNcQ8Dnx0TIs77Z+Z599tnFLrvs0isfWJ5fB2kRlqOwHxi/jPneWzUexngA\nY/Bpu/vWM77ISpniX/zmLRMhXYNxLD4aRbng73G8TjXe0HGuvBGLZTzYMm0mH9qqt5/CXtQNCnsR\nhX2HCSh6FRdZZJHUux3L6PEmP+688870+4wzzkghKwiU6Dlnej32Yd0aa6xR7ktoDmIGkcT/uZBC\nWMW6RmkygJeQGcQ18fDsg+iOc6UXvpEIbHa+rOPhgm15G8B0hKecckoKycnzhYcFQoD6IpYU9v3z\ny9yPBkrA4jeUAf8zGxRjQljGsXKf6K+RNr7D7DT4USeL+vi41vbbb18suOCCqY7we9NNN011jWtR\n2Iso7EUU9l0s7BFTK6ywQrHUUkslUUxPHvlDyAo3enrz+c3g0oixJ0af2WoYjIrQJ42YRxyBw+A+\nxFOsa5Qmg2URRuzD+XEuCHvCclg+a9asdF65sM9FYKO0ebiI8yXtX/7yl+mcEPD5cqYPROhwTQr7\n/vslYrGW0WtOyBPXQA8xb0hYTpn+4Ac/SA9m/KYsY4D2CSecUGy22WYpJIt1jMOIQc9bbLFFelhj\nOf631lprpQHb7HvttdcWe+yxR0qTtzYxEJuHR/5OmjSpPA8eaiONddZZJ+3Hug033DD5109+8pNy\njnp8Ma6Hnm7qTcwpTzqtnh9jQyId1m2zzTbl9bMufuPnRxxxRFn2Rx99dLkdx8uPxZu4ZsfCqFvk\nN+NRSI9teEtWr9wwhb2oGxT2Igr7YRJQ9QwhgrBHROfLIy6dsJZq6AyChQGssW019AWhHAJkv/32\nK5evvPLK5bpGaRIeQfhNrFt11VWTQCE9etVZdv311/c41xgcW+ucGp0vgg7Rw3K+GBrLETaEJbWb\nnwMpeka6sEeEcu6XXHJJekOz2267pXJEKOfCPr7yOmXKlHK2I8oHnw0/Pf7445NYZdBz9DTHOsQs\nDwM8NESaHJsHNx4S42EPUcsA7x133LFYbrnl0naRBjM93XPPPUn0x2DxCy+8MP3Pw21+XTwohM9x\nzFbPLx5UMM4pf7DJf5Mu504IWnyYCt/iTVcci/wkXfKz2bHCIp8x0h0KH1c7iLpBYS+i/w6gsG/H\nCI0hvh2BXV1HLzcPCYiXXJBEbyDWbpr19uNY/T3fWFcrbXrp+5tXCvuefkkZVI1pSulF53/ynOka\nmYYUYY/Axo/4n3EcG220Ubkfohxhz8Pe6aef3uNBEsHKOvIaMRvbsS5Pk/JHYOMDbIcQj+1idid8\nr5pGfi4ck7dJHCu/rjztds4vt/xcq795wGG/yy67LIUSMQCcaWtZznmTHm/g4m0aDwCNjhVGGXAM\ntiPNetuFKexF3aCwF1HYD4OAGkqjV5MeVXoJh/tchtMU9s39EjF68MEH93rbUxX2fBvh8MMPL/fL\nRSpCmxCaWIcYx//Ia9IIgV4VxyxDfJMGy/mf86mmX02jejw+5FYV9nna7Zxfq8Ke89x9993LPFtv\nvfVSD/0555xTc9B4s2OF7bzzzum62Y5Qo8gPhb2Iwl5EYT9KhT29lPSojmZRr7BvzS/pYSakhJh1\nRCQPhbV67On1HjduXLkfISN5j32+LsZhhJiN9OoJe/y1ul1V2OfrOF4u1AkZayTs2zm/3LjG/AGZ\nvIlzpzedtx28ISMdlnNOHIsQIvbByFPGjNS6jqoRWhRhRfGtB0J2FPYiCnsRK+goFvaawr6WX8YM\nRbmddtppSQATAsLAZnqJv/nNb6bfCEwEKwIVUco18jVkwmR+9KMfJeHNPsSzs46PmrEfPfs8LDBO\nI0+D44U45jchLByLqR6J28+343iRfjUNzpm3DHEN9JYzODa/Ls6D62r3/HIjfIb9rrrqqpQOgn31\n1VdP2x5zzDHp3OfOnZvSJByIc7rjjjvSPhdffHEKKWOmJ34zWLfRsXhQiC88xzIeEkLo1yo7TGEv\n6gaFvYjCfhgEVLcY8cfEJ3fzNQyk6OlU3+WrwQPhl4h0BGpcw7777pti3fnyL4KSHuYQogjjGAjN\nbC8I5/gAFOEu+aBnZjiKryLnaeS/6bmmJ519GDSdbxfCnt78ahpMl5oLe0JxGgn7ds4vN86P/Ij9\nVlpppVLYEz/PA0U+uDyOFVPVRl7xkbdGx+I4P/3pT1N6ed1j+cYbb5z2qVcnFfaiblDYiyjsFfY1\nLWbPoVeREAKFfef6LscbM2ZMMXPmzJb9EqFYz2Ja03rrEaeHHXZYKcj5HX4S2yCm6QVvdJxaxn7t\n7tMX6+v58WYBa2cd4T6E30R+DZYp7EXdoLAXUdgPsrAfCpEy0EasMD23TONHr303XsNgiJ5OFvZh\nCy+8cPpOwOeffz5ofhkx3/Ra850E/meg6Ejwk9Hu42oHUTco7EX03wEUUPTqEfcbecCHmaInlIF3\nSy65ZFq+5ZZbppheljOH/He/+920fPPNN08xuiwnLnjNNddMH9Ih5IBBiMwFn28baYQh5OMYhA6w\nfTvpRzqsQ/TFubMufje6xnrn1+hYCvv/9w2A+H/++ecvJk6cmPJ6MB44iRMnxn7GjBltl4WmsBdR\n2CvsxQo6KoQ9YQ0M5iOumDnF4+M08dXY4447LgkpBu8h7mNu7FjOlzgj3jnWIYZZn2/7+OOPp+XE\n7FbDLohRRtQzkBFh0Gr6xEBHGjEAMgR7/rveNUZPcK3za3Qshf3/OyZie5VVVunRiz9hwoT0QaXc\nL2OmFm3kmMJe1A0KexGF/SAL+3ZvzgwcpEear38yKC+m0SO8IgbOsR1CnynxWM4gvugJR/CyP0IZ\nMcf/9HpH2oTYMNiSwX3MSkI+I7Dzc8inB2wn/dyYHhAhH+eb/653jY3Or9GxWhU9teYUH0mWQ37z\n5dZ8PXH4CnuFvdpB1A0KexH9d4iEPQJ3t912K/Ng3XXXTbOSMN3dL37xi17bs/zXv/51+ZtQGkJZ\nEN6IO4Qyy1h39tln1xSEbJunyfYIe0R0O+m3KuzrXWOj82t0LIV97/rCg9jkyZOL+eabr8c24Zes\nHyyL+e950zLcabWz//Tp05PPD2beDKYp7EXdoLAXUdgPsrBv9+ZMCMojjzySZtcgpv773/9+cdBB\nB6U5sJdddtkkjNmOXm6m3+OjNUw3GPvHfODExVdFTaSBsMbomWfaQnrI83MgjW9961tpOfu0mn5u\nfOSIBwCOw28+ksW1sG2za6x1fv0VeKMlFIfY92pPPfH2hOPkvbqDKTAJ2Yqy7m9a+FF/0mp1f2bJ\nIfyMN0UKe7WDqBsU9iL674AI+6OOOioJ5zlz5iThu/zyyyfRizAmT5iphuU///nP0weEENj58kMP\nPTTFrzNlIWI4FzW33npr2vbCCy9MQuakk05KvxkMW0/YE2/favq5xUeMiKdHbCLYOV+2rXeNjc6v\n0bEU9v/fMaux9cyQQ3nlM+Q080sGwxLyxP58eIryD7E+duzYVE6so8wor9iPMmM/jLEfUdbV9BlT\nEYOzeWDENxodt1rutfbn3LbaaqtyG3wvflf3r3ecnXfeOS2LUDDmto9B3Jtttll5nrnxkE3+Rn4f\nccQRDc8zlufp8oAe+bvGGmukAeacH9fQyjko7EUU9iIK+w4W9vQYEpqSx0XHDX3KlCnlcgQIYoDl\n8VXK+OAON/N6YQh5GvRQXnbZZb3OIRf27aafi56999673G/FFVcsxV6r15ifnz32zY8Ztvjii9ed\n0z7ygfKpGkKS/Rm7wRuVXXbZpVhqqaXSm5MY74DwpBc8xDtvVuILq5QdYh8xGuvy9OPrrXyplcHQ\nPOwhehsdl0HXCHPSqrd/vA2K4+W/8/0bHYd1PLAy7oMPdcVx2I6HBLbjYTO/HnyTfQgTiw928Qaq\n3nlGHsZyPqZFPSPdWMex+OIsabZyDrkp7EXdoLAXUdgPsrCvdxNuZgwqxarLEd0I7upyPrZDL3or\naUcaiORWz6ed9Fu5jlausZ3za2ajQdhvtNFG5ew3ffFLBC1vZqJsmGEHMYkoRnh++ctfTg9grI+e\ncNYRQhUPbaxD3MZ+efpsly/neDyANDpuLszr7Z9vw/L8d/5/o+MwdgORTYw9X6aNN0r4IWmQv+Rt\nfj1sR55cdNFFaaA540QQ3vXOs5pP1Cf2J7+q+dvqOSjsRRT2Igr7LhD22sDaSBb2ET/fjl8iLquG\nuD3ggAN69P4jRKPnHOHLNmwbgpl1J598cnHIIYeU6eTr8vSZYWmnnXZq+7iRVr39q8fLf+f/NzoO\nsz1xfQhpRHWtwckI8Op5/+pXvyrX//SnP00ivt55svzAAw8sf3NM3m6QLueJkGcZ61o9h9wU9qJu\nUNiLKOwHWdjXuwlrQ2sjWdgPlF+eccYZKbSEcBpE6+23394jJCb+ryXsCSuJdNivlrBnuxgczW96\nuPfZZ5+0vNFx8+PU2p9yRCDTW149fr5/o+sLYU/veByH9DC2pacf0Z9fD1POEnbDvsTOcxyEe73z\n5PsLhOTkwh4xH8I+z99Wz0FhL6KwF1HYK+wV9gr7XoaYRNwiOBGQq666ahKehK006hWPAdzEmTN+\ngkG20ROep4+QZrtZs2alY9CrzXYMkm7luPX2j/h0Pqg2e/bscqB2rQeDeschlIb/L7744jS9KukR\nPkNYGDNP8ZuBrvn18MVk9mFwN2ny0TiEfb3zRMDny5mOlAcN8qyavzEeoNk5KOxFFPYiCvsRKuyj\nZ08hr7Bv5pcxpWhuxG8jVOMa9tprrzSAmUHaDJiNwZtsW/194okn9gpxiXW58RGyfAA4x2znuLX2\nRwyzT3WgNvvk+zc6DmnQmx4zQOXHYRsG1VavhXh4wm/yWYkQ+fXOMz40F8uJqeehqFZ+VtOodw65\nKexF3aCwF1HYD4OAateYxYP8uPnmm8tl9DD+8Ic/TNMPxrLNN9889QYOxDGrVkt4tGvTpk1LYqjW\nOnoll1lmmWLTTTft1zEGU/SMdGGfC1bKo908pueb3udm29FbXcsPWj1uvf2pE1iz/Rsdh2uoHgfR\n3yi9esetd56t5lM756CwF3WDwl5EYd8lwp6b+rbbblt89atfTaEG/N54441TWAGz1YRY4DfrB0PY\n0+NIqEBfRTfnSa9jPWHPQ0uUO1P9Kew73y+1zjKFvagbFPYiCvtBFlCI8IEwevgQ1sxnfdhhh6XX\n+MzAEesZEEiPN7G6xA7/8pe/THm40korleEKCObVV1+9GDduXNqf2OSrrrqq+M53vpO2/dnPfpYG\nBEaaTAfIdhhhCquttloK9SEd5jHn/5jjPP9NKAPXTpq8RXjyySfTB7X4/Y1vfCOFK1Svj5ldDj/8\n8BQeseeeew5YvoUp7AfHL7XOMYW9qBsU9iIK+y4SUPFl1ogDztchihnMR8gM6+nhR/hvv/32Zc9+\nrEPYH3300eUgQb4Eizhff/31y23jWMT50puO+A9hf9ttt6WHjBDy+e/44mykycMG4p4vfPLGgTm/\nST8/dwYyxhzdDBpkuwg/UNgr7DWFvYjC3sopVtCOEFAxXd1AGB/PQUDTg840e/lUeMzIgRjnq5us\nRyyzjun7+E1PfqyjB511DHgMsZ5vy+wm1XX07C+55JIpNpkvjXIe/M+6/Df7xXasQ+jPmDGjDBWK\n88qN2PuYrYRr5CHi3HPPHdC8U9gPnl8281nKdaiON5pNYS/qBoW9iMK+iwTUjjvumIQ34Sxjxowp\nBRNCnWV8ybIquhnUh1C+4oor0jq2YxnrmP6PD/ZE+rFtCPtJkyaV63goiHQbCXvS5Dyr507aiPd4\nqMgfStZaa61eH+JZd911U8++wr57hX3MVINvDGRZ9tfw1/zhsy82derUNJ6l1jrqZQwE788xFPYi\nCnsrp1hBR6iAolebPGGObKbni3AX1jH3NqE3IVpC5Ieg5jfhMlVBg3jfbLPNeohvtg1hT2hOrONt\nQIh3/ucBIMRavo79EDXR0084EAIPEVSrx543CRFaROgO8fuXXnppWsY5K+y7U9jzhgZfoizjQbJT\nDD/LH0z7cm1f+MIXar59wgg7Cz/AvxX2Igp7hZFYQUeIsEfg9tcIuyE/jjjiiHJZzBuO0GdgKqEr\nLEe8s5yec8Q9ve4h9Nn2e9/7XhLkbHvNNdekbRFfDM5lW+LbESyxjukz+b3hhhum0Bz2DTHOvNqP\nP/54EvKxLj6sE2nSe886hD3nwdSdCKO4DqbrjH1jGYKLjwztscceA5J/mMJ+4P2ykf34xz9O+YAA\nZiwHD24xSHuTTTZJD3xshy8x5zyDryNkLNLgf95M7bzzzuVAcER5vf3qHQM777zzyoHgjPnIfZk0\nwv+qv/H/uBYegvF3PjgVA8H5YFT12sePH58GuDMIfOLEiYOe1wPp42oHUTco7EX030EWUISqrLPO\nOj0EMeIXsY3oQGA88sgjpbBHvIQY4X/ENusQ9vTY5yKaB4D8gznXXnttuS6+dhkWYohjI1hi+Qor\nrNBDnJ9xxhnlOs4NscQ+CKr4QijbIfx50JgyZUqvaz777LPTQ4bCvjuFPWMyEPUXXnhh+ZDI4G58\nIQZp48/xIIqY5g0UD5GRRqzbZptt0luq7bbbru5+8fXbWseIh80zzzwz9abHQPAYJJ4/7Oa/Gcyd\np8kDLOKeBxV8s/qQijHjE/vwAIJ/xRS1CnsRhb3CXqygI0DYI2gH2xAT8T/hK0svvXSK882XNzIE\nNoKq3XXMXBMf+6m1H/H0tZYPRZ5UTWE/tH5J+M3CCy+cfJAHRKZgjYdC/ImHSMQ/D5v8T094NY1Y\nF34c+zHeo7pfo2NU19GzzwMudYT6gpDn/6g/8Zv9YjvWIfSnT5/e49qq50zsPet4Q0bd4CGCMLpu\n8XG1g6gbFPYi+u8wC/vcYmpKehKHQ0B3qinsh9YvmQ0HgYvwJmxsv/3267EOH0V0I6R5q8OyahpV\n0R37XXbZZb32a3QMBPrBBx/c44Eh0m0k7Elzhx12aHht+XIeHNZcc82aA8GHoj4q7EXdoLAXUdiP\nMGHPlJWECijmFfaN/BIROpiGHyJ+H3vsseL4449PM8Tk6xDl9JwT+kKvOMK3mgbrQrzn+xFaU92v\n0TFYt95665XrKHfEO/vGgzA97NV17Bdvv1jH1K3EzRNXz7XxRio/X94kRKgZ34zA+G4Dyzjnwc5z\nhb2oGxT2Igr7LhdQ2tCJHoV934Q9syzFYGuW03tO7DmhLBHTXkvY03vOfieddFIK4WI/xDr/V/dr\ndIxYh+Dm9wYbbJBCc9g3xDgCnK8uI+RjHf6Rp0nvPetietmZM2emsJw43wMPPLDcN5bxUMBA8N13\n311hL6KwV9iLFVRhrynsu1vY8xtxng/SRmxHr3y9HnuEfXUgOGVWb796x8COO+64HuExIcAR3sy+\nFMuXX375HuL89NNP7zEQnF549mEQLcsIC4rr5UGDQd/V62DQLg8ZCnsRhb3CXqygXS7sEQHa8JvC\nfvj9EvFLj3mr2yPe6UEnTIbwl/4eo9E6Zq7B6u1HPH2t5SPNx9UOom5Q2Ivovwp7hb3CfsCNqSm/\n/e1vp3AXfVhhL6Kwt3KKFVQBpSnsa/gloSadbvSg812FbjjXTjCFvagbFPYiCnsFlKJHYa/p42oH\nUTco7EX0XwWUwl5hrynsRRT2CiORUSDsGfinDb8p7PVLfVztIOoGhb2I/quAUvQo7DWFvYjCXmEk\norBXcCjsO88vmWlGG1mmsBd1g8JeRGGvgFL0KOw1fVztIOoGhb2I/quAUtgr7DWFvYjCXmEkMgqE\n/bx587QOMIW9fqmPqx1E3aCwF9F/FVCKHoW9prAXUdgrjERGu7CfO3eu1gGmsNcv9XG1g6gbFPYi\n+q8CStGjsNcU9iIKe4WRyGgX9lpnmcJeH9DH1Q6iblDYi+i/CiiFvcK+yyzKQx9XO4i6QWEvov8O\nKJMnT+5109WG1/Tdke2XUR76uNpB1A0KexH9VwGl6FHYK+wV9iIKe4WRiP7bk2nTpmkdZAr7ke+X\nUR76uNpB1A0KexH9V0TftTzMLxGFvZVTRP8VfVcsD/NLRGFv5RQrqBkh+q5YHuaXiMLeyilWUBF9\n1/KwPMwv0XcV9iL6r4i+a3mYXyIKeyuniP4r+q5YHuaXiMLeyilWUP1X9F2xPMwvEYW9lVOsoCL6\nrlge5pfouwp7Ef1XRN+1PMwvEYW9lVNE/xV9V9+1PMwvEYW9lVNE/xV9VywP80tEYW/lFCuoiL4r\nlof5Jfquwl5E/xXRdy0P88uMEH1XYS+i/4rou5aH+SWisLdyiui/ou+K5WF+iSjsrZxiBRXRd8Xy\nML9E31XYi3hzEdF3LQ/zy/wSfVdhL6L/iui7lof5JaKwt3KK6L+i74rlYX6JKOytnGIFFdF3xfIw\nv0TfVdiLeHMR0XctDzG/RN9V2IvovyL6ruVhfoko7K2cIvqv6LtieZhfIgp7K6dYQUX0XbE8zC8R\nhb2INxcRfdfyEPNL9F2FvYj+K6LvWh7ml4jC3sopov+KviuWh/klorC3cooVVETfFcvD/BJR2It4\ncxHRdy0PMb9E31XYi+i/Ivqu5WF+iSjsrZwi+q/ou2J5mF8iCnsrp1hBRfRdsTzMLxGFvYg3FxF9\n1/IQ80v0XYW9iP4rou9aHuaXiMLeyimi/4q+K5aH+SWisLdyihVURN8Vy8P8ElHYi3hzEdF3LQ8x\nv0TfVdiL6L8i+q7lYX6JKOytnCL6r+i7YnmYXyIKeyunWEFF9F2xPMwvEYW9iBVURN+1PMT8En1X\nYS+i/4rou5aH+SWisLdyiui/ou+K5WF+iSjsrZxiBRXRd8XyML9EFPZWTrGCiui7loflYX6Jvquw\nF9F/RfRdy8P8ElHYWzlF9F/Rd8XyML9EFPZWTrGC6r+i74rlYX6JKOytnGIFFdF3xfIwv0TfVdiL\n6L8i+q7lYX6JKOytnCL6r+i7MuTMnDmzzP9mNmHCBDNM/xV9V2Evov+K6Ludyvzzz9+SsH/jjTfM\nLP1X9F2FvYj+K6LvdiqHHnpoyv9VVlml17qFFlrI3nr9V/Rdhb2I/iui73YDn3/+eVkGr7/+ern8\n3nvvtbde/xV9V2Evov+K6LvdBD3ylMH48ePLZfTgs2zMmDFmkP4r+q7CXkT/FdF3uwF65KMcPvvs\ns9RzH7/vu+8+M0j/FX1XYS+i/4rou90CPfOUw+TJk1PPPf8vvPDCZoz+K/quwl5E/xXRd7sJeuYp\nh/nmm68sE6bDFP1X9F2FvYj+K6Lvdhn00Ed5MA2m6L+i7yrsRfRfEX23C8k/WMU0mKL/ir6rsBfR\nf0X03S4lPljFNJii/4q+q7AX0X9FRo3vLrHEEloHmv4rorC3cooVVETfVdgr7G17xbZXYS+i/4qM\nVmHP/O/a8JvCXkRhb+UUK6iIvtsvYf/pp59qHWAKexGFvZVTrKAi+q7CfgQJ+2nTppV200036b8i\nCnsRxZGIvtuasP/kk0+0DrAoD8R8bvqviMJeRHEkou8q7BX2tr1i26uwF9F/RUaLsP/444+1DjCF\nvYjC3sopVlARfVdhr7C37RXbXoW9iP4rorBXVCvsRRT23lxE9F/Rd7te2H/00UdaB5jCXkRhb+UU\nK6iIvquwV9jb9optr8JeRP8VGe3C/sMPP9Q6wBT2Igp7K6dYQUX0XYW9wt62V2x7FfYi+q+Iwl5R\nrbAXUdh7cxHRf0Xf7Xph/8EHH2gdYAp7EYW9lVOsoCL6rsJeYW/bK7a9CnsR/VdktAv7999/X+sA\nU9iLKOytnGIFFdF3h13Yv/HGG8Vii/3f9s481ooqT8B/TzRpE01GJ3HfO1EnPSAC2iIoaqPiaCsC\n7o3QaivaTNsqCoroBAFBUWlABUF2BIFhBwUUWQXZlMUdt15sdUz6n/6jJt/JnEq9++6977713vve\n9yUnUHvVqd+p+u55p875eZonMfXp0yf54IMPStrHtGnTkgMHDij2ir2IYm/hFAuoiLFbTrE//vjj\nkzfffDN5//33kx07diRvv/12ctVVVyXXXHNN8vXXXxfd/tChQ8nhhx+eHDx4ULFX7EUUewunWEBF\njN2GiP0333zT6PTpp58mJ598cvLxxx/XmD958uSkQ4cOyZdffhmm16xZk5x22mkh33r37h1q85n/\nu9/9Lszjx8Hq1auTm2++Ofnqq6/Css2bN6fTW7ZsSS677LLklltuSY466qhkwYIFya9//etk6NCh\nYfvzzz8//LjIPb/c7dhnoXMhLVmyJDn33HPDshtvvDFdtnTp0hrb7Nq1q0nyj6TYiyj2Fk6xgIoY\nuxUh9kj5woULQ039+vXrk9mzZ4d5/fr1C+vs3Lkz5NeoUaOS7du3B8lG+qmtR7L/7d/+LZk3b17y\n1ltvhWY9UezZV5x+9913wz7YdvTo0WE7pm+99dZwXCS8e/fu6bYx5W63cePGgueybdu2dBk/ADp1\n6hT2G/cR5/OXCH7MsI1iL6LYWzhFjF8xdssu9jSTaWz65JNP8raxf+ihh0ItPutMmDAhiP7evXuT\nffv2BRFnHST7888/D5K8f//+5J133gn7opaf7bLTGzZsCDXu1KCzLN90dtuYctcrdi7PP/98+lcG\n1kX0Z8yYEebHHw3M/+ijj8I+ly1b1iR5qNiLKPYWTrGAihi7FSH2iPnu3bvD9PLly0PejB07Nl1n\n4sSJtcSfhBjH7RHvYmLP/xFy1o/L2O6LL76otW72/HK3K3YuL774YnLXXXfVukbmDxkypMY10yxH\nsRdR7C2cIsavGLutTuxjjXhWnmkzzzQ13jRroekKCRlftGhR2CZuT409TW8QZtZhu3Xr1tUQ+2xt\ner7pQmKfXa/YucRlsWaej4H5y8OYMWOSvn371rhmfiwo9iKKvYVTxPgVY7cixB6BbWyiuc1JJ50U\nmrbEeQhz586dk/bt24emNgg++UWzFnrRoTafaT5ApTccJHnOnDlps5j58+cne/bsCZJ9ySWXhP2x\nDHHn/xyjrumYcucXO5e1a9eG/7/xxhvhuu68885wfD6czc4fPnx4cswxx4QuOpsiDxV7EcXewikW\nUBFjtyLFnkRNOXk0adKkMM2/Md/o3nLu3LlhPjXpfKDKfD7AfeCBB9L1fvnLX9YQ+/iRaxT2YtNZ\nsc+dX+hccpfxg2PTpk1hPs1x4nza169YsaJJ8k+xF1HsLZxiARUxdhst9kh1SyaasNDkBcnOXcYP\nhPh/avFJ5TqXuCzfOVJL39TnotiLKPYWTrGAihi7VSX2JsVeRLG3cIoFVMTYbQaxjx+QmsqbFHsR\nxd7CKRZQEWNXsVfsffaKz17FXsT4FVHsq1eGaevOqLeKvc9e8dmr2IsYvyJtXuzpw70aU+w558wz\nzwxyX63XEZNiL6LYWzjFAipi7LY5sadnmuOOOy6ZN29eqLWvdqlX7EUUewunWEBFjN1Giz2DR9Un\nMaDTsGHD0utnoCZqzFnG4E3t2rUL83v37p3s3LkzzKdv+lNPPTXM79WrV7Jt27Ywn37uu3fvntx0\n002hX3j6m1++fHmNdeM+YmJ02ngM+p9n/frsP+6HZfSdH8+dZXG62DUWOr9ixyolKfYiir2FUyyg\nIsZui4r97Nmzw4ir69evTxYsWBDyYNGiRcnGjRvD/0eMGBGktmPHjkHu161bV2N+z549w2BW1LrH\nZcgwy7Prbt68Ocw/99xzw7rZc1i8eHGQ+lmzZgX5LXX/9Esf97FmzZq0GU/udKFr3Lp1a8HzK3Ys\nxV5EsbdwigVUxNitOLFn9FVqpGfMmBHkFdlFbJ999tkguVGUGa116tSpYf7FF18casGZ/+GHH4bt\nkXNkmP/v2rUr3TdNbKgF3717d/LWW2+FPM6t/f7oo4+CvO/Zs6de+8+mtWvX1hD77HShayx2fsWO\npdiLKPYWTrGAihi7zS72CHF9ElJ9zz33pNffo0ePIPHPPfdcMmDAgFrrM3/w4MHpNE1paMpCDThi\njCgzj2WIc9xvNrFudp+sj9gj2PXZfzaxDJFnhNnc6ULXWOz8ih2rlKTYiyj2Fk6xgIoYuy0q9lu2\nbAnNbhBY2rYjw4j12LFjk/POOy9to04tNz3XjBw5MjTJyYo3AhxlmFr+KNdxH9SOkxDs+fPnB4Ev\nJPZsU+r+s4mmN/wA4DhMr169OhX7uq4x3/kVO5ZiL6LYWzjFAipi7Da72CPi9UlPPvlkEOft27cn\n+/btSzp16pQ8/PDDyapVq0J+zJ07N8zv379/0q1btyDG2fmPPfZYaL9OM5o333wzOeOMM4Iks+9l\ny5aFdWnCQ283o0ePDtN8DJs9B/Zz4oknJu+//35oclPq/rOJ9vNsR3v6HTt2BGHnfFm30DUWO79i\nxyolKfYiir2FUyygIsZui4o9bchpmhKvv0uXLkGMWTZ+/Ph0PmJM23Pm01wmzqcdOsLKfGQ4fnwa\n95/dBx/Izpw5s9Y5ZMW+vvuPiZr1QYMGpdudf/75qdiXeo3Z8yt2LMVeRLG3cIoFVMTYrTixj4mP\nSkm582m+QtOUfDK+d+/ekvYd94F8l3o+9dl/KddRyjXW5/wUexHF3sIpFlARY7dZxR45NZU/KfYi\nir2FUyygIsauYq/Y++wVn72KvYjxK9LWxT728GIqb1LsRRR7C6dYQEWM3aoRe3qSoX26Iq/Yiyj2\nFk6xgIoYuxUo9nT/SF4sXbo0nUd3k2effXboJjLOu+GGG5I5c+Y0ixjTL3379u2TgwcPNngfkyZN\nCj3t5FvGDxK6xLz++usbdQzFXkSxt3CKBVTE2G0WsWeQpcYmRPfmm29Ojj766OS9994L09dcc03o\nkvLDDz8M69BjDdMsb4pj5qa1a9eGfuT5q0BDtuc86b6Sbi3zLV+yZEl6zxmMqqnPX7EXUewtnGIB\nFTF2yy72pN27dwex7tu3b/L4448nRx55ZPLuu++my5cvX5506NAh9Pd+0UUXJXfeeWfIvwsuuCBZ\nt25dKucXX3xx2AfbI9Cvv/56csopp4R1r7vuumTTpk3pPqdMmRLWI/HXAPqhR+zZT58+fVLJZz/Z\n6Xnz5oVrZ5+9evUKA0wNGDAg7X8/34+P2267LXniiSdC3/e///3vFXsRxd7CKWL8irFbWWJP7XpT\npTgyaxwJNrts2LBhyfDhw5PVq1eH5dTwb9iwIbn99tvTmv24DAl/6qmnkkWLFoVpRoLlB8GVV16Z\nrhuP9cILL4TadOS/a9euockMPyL4kcH/OXZ2Oo44G/fJjw3knhFs+YvDtGnTwv6z584ItGzDDwaa\nG7EeNftNmXeKvYhib+EUC6iIsVsxYs+IrQg0NehIe5xPTXnHjh2DYCPvLI9izEBPTNNGPi6jBp1l\no0ePDrLO9tl158+fX2sZNfu0sUfeV61aVUPss9NsF9djGaPjvvLKK2EgKn405BN22t7HHxRcIz8i\nXnrpJcVeRLG3cIoYv2LsVo7YI8ZNlfr16xfE+9hjj026dOkSZJn5W7duDfP4oHblypVBshkplmU0\n4UGUZ8+eHZaxHvNY9swzzyQPPvhguv+4LhI/atSo5NFHH02XRXlnv7nHyE6zT84z99zZN/LOuWbn\n8wPg0ksvTe93TD169Aii31R5p9iLKPYWTrGAihi7FSH2EydODPmxYsWKUAsem7uw7LXXXgtNb6Jk\nR8mPQs00tfksa9euXSrkyDu90GTlm3Wj2NM0Jy6jaU6Ud/7PD4Ao3tllbEfzG4SdZfxlgTbztKtH\n7Gl2k70u/pIQmxbRdIf2+jNmzAjzOGfFXkSxt3CKGL9i7LYasUeOyQs+Lo3zaPLCPGSdD1NpzhJr\n1pk/ZsyY0FPOkCFDgqzz/9ya9gULFoR1Z86cGaSedWP79rgM4Wb66quvDk1z2DbKOAK+ZcuWIPJx\nWWybH/dJ7T3LEHrOg647418aSIMHD063zdbi07Ro4MCBir2IYm/hFDF+xditDLFHUhubaKpy+eWX\nBzmP8xDhnj17Jr/4xS+CMNNDDvORd5rrMJ/84/98kMoyavupsaemPe6HpjMxr1kXoY/LRo4cWaN5\nDALOthz7vvvuS+d37tw5XcZ248aNS5dxbvwQYBs+omXerFmzwnq0pz/99NOT8ePH17rmF198MfzI\naIr8Iyn2Ioq9hVMsoCLGbtnFvq5EM5f4f+SdGnQkOzu/WKJmnVr5+i5jsClSoe34SDff/JbIE8Ve\nRLG3cIoFVMTYbVKxp6a6JVPsmpLa/ZY+diUnxV5EsbdwigVUxNitKrGnBp3+6ZV5xV5EsbdwigVU\nxNitYrE3KfYiir2FUyygIsZuM4g9bd1N5U+KvYhib+EUC6iIsavYK/Y+e8Vnr2IvYvyKtHWxp892\nU/mTYi+i2Fs4xQIqYuwq9oq9z17x2avYixi/Ioq9Uq3Yiyj2vlxEjF8xdqte7OlP3lT+pNiLKPYW\nTrGAihi7ir1i77NXfPYq9iLGr0hbF/s9e/aYKiAp9iKKvYVTLKAixq5ir9j77BWfvYq9iPErotgr\n1Yq9iGLvy0XE+BVjt+rFfvfu3aYKSIq9iGJv4RQLqIixq9gr9j57xWevYi9i/Iq0dbHftWuXqQKS\nYi+i2Fs4xQIqYuwq9oq9z17x2avYixi/Im1d7Hfu3GmqgKTYiyj2Fk6xgIoYu4q9Yu+zV3z2KvYi\nxq9IWxd7U2UlxV5EsbdwigVUxNhtk2If74Ni77NXfPYq9iLGr4ix+/8MHTq0llhWeor3odrOu9Rk\n/Ioo9iLKkYixq9gr9j57xWevYi9i/Iq0xdidOHFi1aV4H6rx3OtKir2IYi+iHIkYu94HMd/E2FXs\nRYxfEWPX+2C+iSj2Fk4R41eMXfE+mG8iir2FUyygIsaueB/MNxHFXsQC2uI4oE5lJmPXZ4iYb2Ls\nKvYixq9ir9j77PUZYr6JKPYWTpG2Kvbff/+9qQKSYu8zRMw3MXYVexHjt1Fi//e//91UAUmx9xki\n5psYu4q9iPGr2LcisW+JQX589voMMd9EFHsLp0grFPvvvvvOVAEp3g9kPpuMXZ8hYr6JsavYixi/\nir1i77PXZ4j5JqLYWzhF2orY/+1vfzNVQFLsfYaI+SbGrmIvYvwq9oq9z16fIeabiGJv4RRR7JVq\nxV58hphvIoq9hVMsoFUv9n/9619NFZAUe58hYr6JsavYixi/ir1i77PXZ4j5JqLYWzhF2rrY/+Uv\nfzFVQFLsfYaI+SbGrmIvYvwq9oq9z16fIeabiGJv4RRR7JVqxV58hphvIoq9hVMsoFUv9n/+859N\nFZAUe58hYr6JsavYixi/ir1i77PXZ4j5JqLYWzhF2rrYf/vtt6YKSIq9zxAx38TYVexFjN8WF/vP\nPvss+fnPf57mQUx9+vRJPvjgg2YR3/Xr1yennXZa8tVXXyn2Pnt9hphvZoQYu4q9iPHbVGJ//PHH\nJ2+++Wby/vvvJzt27Ejefvvt5Kqrrkquueaa5Ouvv25y8T148GCybNkya+yNXZ8h5pv5JsauYi9i\n/OYT+2+++abe6dNPP01OPvnk5OOPP64xf/LkyUmHDh2SL7/8MkyvWbMm1LKTT7179w61+XHdJUuW\nJOeee25YduONN6bLCm2zbdu25L777gs19oMGDUpmzJiR7mvlypVhHxy32DFj2rJlS3LZZZclt9xy\nS3LUUUclmzdvbtC5Ll26tMY2u3btalB+khR7nyFivomxq9iLGL9lEXtq7BcuXBhq6mkmM3v27DCv\nX79+YZ2dO3eG/Bk1alSyffv2INFI/6FDh4Kkx2VIdadOnYIwF9uGY9D8B7F//vnnwzb8n2Pdcccd\nyR//+Mei22fP/9133w3rsXz06NHJxo0b632ucR9xPn+p4MdO7rEUe/E+mG8iir2IL5cWEXuazdQ3\nffLJJ3nb2D/00EOhFp91JkyYEER/7969yb59+8IPANZBohHzWLPPusgzNfDFtnnnnXfCMdmG5j/M\nR8I/+uijUOuO+BfbPnv+GzZsCNtQ897Qc2V+9+7dw48L5sfzoLlQQ/JUsfcZIuabGLuKvYjxWxax\np3Z69+7dYXr58uUhL8aOHZuuM3HixFriT0J8X3zxxeSuu+6qtd9i22TFHpmmPT/r81cDmsN88cUX\nRbfPHod9IfJcR0PPlflDhgypkSech2Iv3gfzTUSxF7GAVp3YxxrvrByvXr06TMfmMjRNISHeixYt\nCttkm9KwLjXw1PaPGTOm4DZZsWebWbNmJf/+7/+e9OjRI3n22WfrPGau2Gdr4Rt6rn379q2RJ/xY\nUOzF+2C+iSj2IhbQsog9wlrfRHObk046KTRdifMQ4s6dOyft27dPPv/88yD45A/NVuhFh9p8pvnA\ndO3ateH/b7zxRtjXnXfemVxyySXJqlWrCm6zbt26IPYch+PRZAaRPvzww0OTHOYVO2b2/Glqk91X\nQ86VD2ez84cPH54cc8wxyYEDBxqUp4q9zxAx38TYVexFjN+KEHsSNeHkyaRJk8I0/8Z8QsDnzp2b\nrptdhqBv2rSp6DbIePygNe7jD3/4Q+iNJjbPqeuYWbHP3VdDzpXmOHE+7etXrFjRoPxU7H2GiPkm\nxq5iL2L8NlrskeLmTDRRoUkLEl1oWX22aewxm/pc+ZFDLX1j80mx9xki5psYu4q9iPFb0WJvUuzF\n+2C+iSj2Ir5cWkDs4wejpvImxd5niJhvYuwq9iLGr2Kv2PvsrTLiWAz07CTGrxi7ir2I8dvmxZ62\n7YyCq9j77K02pkyZEu7B1VdfbWYYv2LsKvYixm/TiD19tldjeuCBB0L+n3nmmUHuq/U6Yqqv2COG\nPnurly5duqSDuonxK8auYi9i/LZZsacnmuOOOy6ZN29eqLWvdqkvVex/+OGHIPQnnHBCo2LPZ295\niWMjkBg3QYxfMXYVexHjt0nEnsGk6pMQkWHDhqXXz8BM1JizjMGa2rVrF+bTv/zOnTvD/IULFyan\nnnpqmN+rV69k27ZtYT793nfv3j256aabQj/w9C+/fPnyGuvGfcS0f//+9Bj0N8/69dl/3A/Lbrzx\nxvTcWRani11jofMrdqxSUjGx53wee+yx5IgjjkjPSbGvTvhxRrt68v/WW281Q4xfMXYVexHjt3xi\nP3v27DDC6vr165MFCxaEPFi0aFGycePG8P8RI0YEqe3YsWOQe0aMzc7v2bNnGNyKWve4DBlmeXbd\nzZs3h/nnnntuWDd7DosXLw5SP2vWrCC/pe6ffujjPtasWZM248mdLnSNW7duLXh+xY7VULF/7rnn\ngvxlZf5nP/uZYl+l8AMtSj0/0pB8MX7F2FXsRYzfsok9o61SIz1jxowgr8guYvvss88GyY2izOis\nU6dODfMvvvjiIDXM//DDD8P2yDkyzP937dqV7psmNtSC7969O3nrrbdCHufWfn/00UdB3vfs2VOv\n/WcTzSGyYp+dLnSNxc6v2LHqK/aPPPJIctZZZ9UQeka8HTp0aPL9998r9lUGPeBkf6Ah9cwT41eM\nXcVexPhtUrFHiOuTkOp77rknvf4ePXoEiad2ecCAAbXWZ/7gwYPTaZrS0JSFGnDEGFFmHssQ56zM\nxsS62X2yPmKPYNdn/9nEMkSeEWVzpwtdY7HzK3asUhL3grbz//qv/1pj3+ecc04yefLkvLFnqr7E\nR7O2q9cdxNhV7EV8uVSE2G/ZsiU0u0FgaduODCPW9O5x3nnnpW3UqeWm55qRI0eGJjlZ8UaAowxT\nyx/lOu6D2nESgj1//vwg8IXEnm1K3X820fSGHwAch+nVq1enYl/XNeY7v2LHKlXsjzzyyOSwww6r\n0exmzJgxoZZesa/exA82auz5q5D47BVjV7EX8eXSbGKPiNcnPfnkk0Gct2/fnuzbty/p1KlT8vDD\nDyerVq0K+TF37twwv3///km3bt2CGGfn8xEo7ddpRvPmm28mZ5xxRpBk9r1s2bKwLk146O1m9OjR\nYZqPYbPnwH5OPPHE0JyBJjel7j+baD/PdrSn37FjRxB2zpd1C11jsfMrdqxSUrwfnM8dd9xRo+Ye\nwR84cGBYryliz2ev+OwVUewtnCKKfWhDTtOUbNMCxJhl48ePT+cjxrQ9Zz7NZeJ82qHTjpz5yHD8\n+DTuP7sPPpCdOXNmrXPIin199x8TNeuDBg1Ktzv//PNTsS/1GrPnV+xY9RH77Mez999/f9rfeUy3\n3HKLYi8+e41fMXYVexHjt/FiHxMflZJy59N8haYp+WR87969Je077gP5LvV86rP/Uq6jlGusz/k1\nROxjd5c04cjtHUexF5+9IsauYi9i/NYQe+TUVP5UygBVtMW/77770v7sffaKz14Rxd7CKWL8KvZV\nKPaROAKtz17x2Sui2Fs4RYzfVCRjDy+m8qb6iL3PXvHZa/yKsavYixi/ZRV7epKhfboir9iLGL9i\n7Cr2IsZvBYo93T+SF0uXLk3n0d3k2WefHbqJjPNuuOGGZM6cOc0ixvRL3759++TgwYMN3sekSZNC\nTzv5lvGDhC4xr7/++kYdQ7EXMX7F2FXsRXy5NIvYM8hSYxOie/PNNydHH3108t5774Xpa665JnRJ\n+eGHH4Z16LGGaZY3xTFzEz3H0I88fxVoyPacJ91X0q1lvuVLlixJ7zmDUTX1+Sv2IsavGLuKvYjx\nW3axJ+3evTuIdd++fZPHH388jKL67rvvpsuXL1+edOjQIfT3ftFFFyV33nlnyL8LLrggWbduXSrn\nF198cdgH2yPQr7/+enLKKaeEda+77rpk06ZN6T75gJT1SPw1gH7oEXv206dPn1Ty2U92et68eeHa\n2WevXr3CAFMDBgxI+9/P9+PjtttuS5544onQ9/3vf/97xV7EZ6+IYm/hFAtoZYk9tetNleLIrHEk\n2OyyYcOGJcOHD09Wr14dllPDv2HDhuT2229Pa/bjMiT8qaeeShYtWhSmGQmWHwRXXnllum481gsv\nvBBq05H/rl27hiYz/IjgRwb/59jZ6TjibNwnPzaQe0aw5S8O06ZNC/vPnjsj0LINPxhobsR61Ow3\nZd4p9iLGrxi7ir2I8VsxYs+IrQg0NehIe5xPTXnHjh2DYCPvLI9izEBPTNNGPi6jBp1lo0ePDrLO\n9tl158+fX2sZNfu0sUfeV61aVUPss9NsF9djGaPjvvLKK2EgKn405BN22t7HHxRcIz8iXnrpJcVe\nxGeviGJv4RQLaOWIPWLcVKlfv35BvI899tikS5cuQZaZv3Xr1jCPD2pXrlwZJJuRYllGEx5Eefbs\n2WEZ6zGPZc8880zy4IMPpvuP6yLxo0aNSh599NF0WZR39pt7jOw0++Q8c8+dfSPvnGt2Pj8ALr30\n0lqjvvbo0SOIflPlnWIvYvyKsavYixi/FSH2EydODPmxYsWKUAsem7uw7LXXXgtNb6JkR8mPQs00\ntfksa9euXSrkyDu90GTlm3Wj2NM0Jy6jaU6Ud/7PD4Ao3tllbEfzG4SdZfxlgTbztKtH7Gl2k70u\n/pIQmxbRdIf2+jNmzAjzOGfFXsRnr4hib+EUC2irEXvkmLzg49I4jyYvzEPW+TCV5iyxZp35Y8aM\nCT3lDBkyJMg6/8+taV+wYEFYd+bMmUHqWTe2b4/LEG6mr7766tA0h22jjCPgW7ZsCSIfl8W2+XGf\n1N6zDKHnPOi6M/6lgTR48OB022wtPk2LBg4cqNiL+OwVUewtnGIBrQyxR1Ibm2iqcvnllwc5j/MQ\n4Z49eya/+MUvgjDTQw7zkXea6zCf/OP/fJDKMmr7qbGnpj3uh6YzMa9ZF6GPy0aOHFmjeQwCzrYc\n+7777kvnd+7cOV3GduPGjUuXcW78EGAbPqJl3qxZs8J6tKc//fTTk/Hjx9e65hdffDH8yGiK/CMp\n9iLGrxi7ir2I8Vt2sa8r0cwl/h95pwYdyc7OL5aoWadWvr7LGGyKVGg7PtLNN78l8kSxFzF+xdhV\n7EWM3yYVe2qqWzLFrimp3W/pY1dyUuxFjF8xdhV7EeO3qsSeGnT6p1fmFXsR41eMXcVexPitYrE3\nKfYixq8Yu4q9iC+XZhB72rqbyp8UexHjV4xdxV7E+FXsFXufveKzV0Sxt3CKtHWxp892U/mTYi9i\n/Iqxq9iLGL+KvWLvs1d89ooo9hZOEcVeqVbsRXz2iij2Fk6xgFa92NOfvKn8SbEXMX7F2FXsRYxf\nxV6x99krPntFFHsLp0hbF/s9e/aYKiAp9iLGrxi7ir2I8avYK/Y+e8Vnr4hib+EUUeyVasVexGev\niGJv4RQLaNWL/e7du00VkBR7EeNXjF3FXsT4VewVe5+94rNXRLG3cIq0dbHftWuXqQKSYi9i/Iqx\nq9iLGL+KvWLvs1d89ooo9hZOkbYu9jt37jRVQFLsRYxfMXYVexHjV7FX7H32is9eEcXewinS1sXe\nVFlJsRcxfsXYVexFjN82KfbxPij2PnvFZ6+IYm/hFDF+/5+hQ4fWEstKT/E+VNt5l5qMXRHjV4xd\nxV7E+FXsFXufveKzV0Sxt3CKtMX4nThxYtWleB+q8dzrSoq9iPErxq5iL2L8eh/EPBPjV0Sxt3CK\nGL/eB/NMxPgVUewtnGIBFe+DeSZi/Ioo9iK+XMT7YJ6JGL9i7Cr2Isav90HMMzF+RRR7C6eI8et9\nMM9EjF8Rxd7CKRZQ8T6YZyLGr4hiL2IBFe+DeSZi/Iqxq9iLGL/eBzHPxPgVUewtnCLGr/fBPBMx\nfkUUewunWEDF+2CeiRi/Ioq9iAVUvA/mmYjxK8auYi9i/HofzAjzTIxfEcXewili/HofzDMR41dE\nsbdwigVUvA/mmYjxK6LYWzjFAireB/NMxPgVY1exFzF+vQ/eB/NMjF8Rxd7CKWL8eh/MMxHjV0Sx\nt3CKBdT49T6YZyLGr4hib+EUC6h4H8wzEeNXjF3FXsT4Fe+DeSbGr4hib+EUMX69D+aZiPErothb\nOEWMX++DeSZi/Ioo9hZOsYCK98E8EzF+xdhV7EWMX/E+mGdi/IoYu4q9iPHrfTDPzDMxfkUUewun\niPHrfTDPRIxfEcXewlk627ZtS956661kwoQJIfXv379eKW7HPtiXGL/eBzHPxPgVUewtnM3M/v37\nkxkzZgQhv+KKK5L/+I//aJbEvjkGx+KYYvx6H8wzEeNXRLG3cDaCn376Kcj1oEGDkgsvvDCvhPfq\n1SsZOHBgMm7cuJCWLVuWvP322yUl1o3bsQ/2le8YHJtz4Fw4JzF+vQ/mmYjxK6LYWzhLYPHixaHG\nPFewr7/++uThhx9O5syZk2zfvj353//932ZJ7JtjcCyOmXsenBvnKMav98E8EzF+RRR7C2cOX3/9\ndTJq1KhaNfO333578sorryQHDhxoNpGvK3FszoFzya3J55w5d/Hl4n0wz0SMXxHFvk0XTj5YpZlL\nVph/9atfJS+//HLy1VdfJT/++GNFJc6Jc+Mcs+fMNfjxrS8X74N5JmL8iij2ba5wUsudK/TUiC9Z\nsqTiZL5Q4lxza/G5Jmvwfbl4H8wzEeNXRLFv9YWTj0+HDh1aQ4Zpy07vM9Ui9LmJc+castfENfqh\nrS8X74N5JmL8iij2rbJw0ld8tg39vffem+zbty/54YcfWkXiWrimbBt8rtkC6svF+2CeiRi/Iop9\nqyic1Fxne7mhffratWtbjdDnJq4t2wafa2/Ltfe+XLwP5pmI8Sui2LeCwkkzlWwt/UsvvZQcOnQo\n+f7771t14hq51mztfVsd7MqXS8szZcqUNN/rSrfeeqsZZuyK8Sui2Fs4i5NtS08N9tatW1u90Ocm\nrjlbe0+eWEClJTjiiCNKEvvPPvvMzDJ2xfgVUewtnPnJbXrz0EMPJV988UXy97//vU0mrp08aKtN\nc3y5lIfHHnss5Dt/Lcrl+OOPt7be2BXjV0Sxt3DWLfW9e/cOAvvLX/4ymTlzZpsV+txEXpAn5A15\n1Fbk3pdLeeB7j5j3n376aTqfD7qtrTd2xfgVUewtnEXJtqdHYLds2aLQ5yTyJMp9W2l378ulfFAj\nT97fcsst6TzijnldunQxg4xdMX5FFHsLZ3Gpp005Avvdd9+Z8iTyJra7bwty78ulfFAjH/Ofbz6o\nuY/T9N4kxq4YvyKKvYWzBoy0GqX+uuuuSz7//HMFvo5EHpFXUe5b82i1vlzKCzXz5D8fblNzz/9P\nOOEEM8bYFeNXRLG3cNYk26YeUaWG8G9/+5uphEReRblvzW3ufbmUF2rmyf+f/exn6b2gO0wxdsX4\nFVHsLZx5pZ5243v27FHY65nIs9jmvlevXq1S7n25lB9q6ON9oBtMMXbF+BVR7C2cBaV+8+bNinoD\nE3nXmuXel0v5yQ5YRTeYYuyK8Sui2Fs4UwYNGpRK/aZNm5K//vWvpkYk8jDK/cCBAy2g0uTEAavo\nBlOMXTF+RRR7C2dg1KhR6WBLa9asUcybKJGXMV+ffvppC2iFEO+JqbKSsSti/Iqxq9g3ksWLF6cv\n1unTpyd/+ctfTE2YyNOYv2+88YYFVLE3KfYixq8Yu4p905Ptq/6JJ55QxJspkbfkcadOnZK9e/da\nQCtE7OkH3lT+pNiLGL9i7Cr2jST7sSyjWSrgzZvIY/L6iiuuCDJjAS2/2DtycmUkxV7E+BVjV7Fv\nJLFdPR947tq1K/nzn/9sasZEHsePaflQ2QKq2Jtqiv3EiRPT9D//8z/GrojxK8auYl8K27Ztq9Hu\nW/FumURex3yfP3++BbTMYu+IyZWR4v1A5rPJ2BUxfsXYVexLgOYgvEgfeOCB5NtvvzW1YCLPyXu+\nbWDaAqrYK/aKvYjxK8auYt8gJkyYkDbB+eijj5TtFk7keWySc9dddyX/+Mc/LKBlEnsHU6uMpNiL\nGL9i7Cr2DeDrr7+u0RTkm2++aZakwBdPCxYsSO/DjBkzLKCKvWKv2IsYv2LsKvb1Y+jQoWkvOM0l\n9abSUuwl58Ybb0wOHjzoy0WxV+wVexHjV4xdxb40srX1K1euVK7LnLgH8X48//zzVdUkp7WIvaMj\nV0ZS7EWMXzF2Fft6EmvradeN5JvKn7gX8S8oy5Yt8+Wi2Cv2ir2I8SvGrmJfnGxt/Y4dO5TqCknc\ni3hfxo0blxw6dMiXSwuKvQOnVUZS7EWMXzF2Fft6EAejoob4q6++MlVQirX2/Dtz5kxfLoq9Yq/Y\nixi/Yuwq9vn56aefQp/pvDiXL1+uTFdY4p5wbzp16hRG3KyGD2kVe5NiL+KzV0SxL0PhXLx4cXhp\nXn755Yp0hSbuDfdo+PDhyfTp05N//vOfvlxaQOwdDbkykmIvYvyKsavYl0jv3r3DS/PFF19Mvvzy\ny6KJwZPOPPPM9PxIZ511VjJ37tx0nbfeeis57bTTks8//7zO/eVLb7/9dtKhQ4fkiy++aND2LZGa\n4hynTJmSfPDBByWty73hHvXo0SN57bXXKr7WXrE3KfYiPntFFPsWLpzZj2b37dtXktgff/zxoZb/\nnXfeCYL73HPPhXOdPHlyWOfDDz8MyxsqvGvXrg0/HipZ7Ddu3Nioc/z000+Tww8/PORVKetzb+J9\nQuwrva19axF7B0urjKTYixi/Yuwq9iXAqKbxw0x6XKkrUVN80kknhZrm7HxqlBF+lm/evDm59957\nQ4094vvEE0+k1/Pkk0+G+e+++27SrVu35J577gnzaeOPLLOv9evXB2lmPaap2T7qqKPCeuecc06y\ndOnSMP/+++8P/bvHc3jzzTeTq6++OkhznFfsOCzr3r17ctNNN4X9b9iwIfwg4a8NrNurV69k+/bt\n6b5oAsN6pL59+yaXXHJJei3sI54v+8lOL1q0KGnfvn3YZ58+fZLdu3eH/GaaPGO6lLyPH9EOHjw4\nefXVVyu6hxzF3qTYi/jsFVHsW7hwDho0KG2GUx+xz5VRZBbhZT5NcaKY00TnmGOOCTX7CxcuDNfE\nC5lp/n/bbbcl27ZtS+64446wX6Q8K/arVq0K61FLvXXr1iDo5557bvLZZ58l48ePT7fhHP7rv/4r\nGThwYI3zKnacuAwJHzlyZLJixYow/fTTT4fr4UdCXJcfDSybNGlSOCfkP4p99no5ZnaaHznZfXbs\n2DHIPYNPkS+zZ8+u8UOkWIrNcfhRMX/+/Iru116xNyn2Ij57RRT7Fiyc9IYTX5jvvfdeo8QeSY5i\nnxVz5Jv5NB2hOQlyHaWa+bHmf//+/WGa5j3Z7bds2ZIsWLAgrHPgwIFk2rRpQexZtnPnzpBHbMN5\nUfuNdOc7r3zHyZ4zy2hShKzzV4bsughE7jLmxfPI/QtDdprt4nos43r4kfLJJ5/k/ctHscQ9ivdr\n3rx5YT+VOhptaxF7R0CujKTYixi/Yuwq9nVAzTIvy2uvvTYIaykJuUZId+3aVWM+NdpIMPPXrVsX\nxJZadQSWZjnxevj4E7nNrhP3e+qppwaJz92epifZj3UvvvjidDuay9AHP7XtnBfrZ8+rruMcd9xx\nYR7LGADqkUceqXGtrItAPPvss8njjz+eLovyzn5zj5GdZp+//e1vS87HuhL3invGXxhojkMbfV8u\nir1ib+yKGL9i7LZxsY+DUg0dOrTeYr93794a82nvTs00Yp0VW5rP0ByFGnXarzMfeY5Szce4cb9M\n82Mjuz3NT2iysnr16rDvNWvWpE1x2I6mPrS7R/Bpy597vnUdJ7sv5J1mMtlrZd0o9jTNics4j3iO\n/KjhBwB/ichdxnY0v4nfG/AXiz/+8Y/Jnj178uZjXYl7xT27++67Q9OmOXPm+HJpRrF39OPKSIq9\niPErxq5iXwf9+/cPL0vkEPEsJdE8BdmdNWtWqCVHLu+7775wrkg260SxRXSfeuqpsP77778ftmWQ\nJWrgkWq2oUYb6R42bFgq4LFXHLZHjBFgtqV2u2vXrmE9pjkWzXuQavZF05rc8y31OKy7ZMmSsO7r\nr78e9s+6/KiguUxc9sYbb4Tpa665JvzlINtWn+uneRAiH5fxgyS7zwEDBoRlCD3nQROljz/+uOT8\n517Fv7IgNtTaV2JzHMXepNiL+OwVUexbsHDGlyW16vUR+9x+7JF1PgKN6yDM1IQjttRMX3HFFem6\nF110UZBfhJumO+3atQvz+T8SnLs9beER4Lg9H8jSTSQ928TjDRkyJAg/Nfr5xL6U48T1+QEQj8W6\nCH1cNnbs2FpNgtiWmnnOK86/4IIL0mVs96c//SldxrVwTWzDXweYx4ewpeY/9yorOS+//HI4ji8X\nxV6xN3ZFjF8xdtuo2MeXJRJaqlQ2JlGzTsrKPzXbUf7r2p518tVsI8j8cOAj3Xzb1fc4JJrg0Ha9\nvstyrzGbaIrER7r59lffvOSexf7s+VFADzu+XJpH7B31uDKSYi9i/Iqxq9gXgbbevChvvvnmIMct\nnag1pwkNst7QfSxfvjzkEc1lkPbmOk6lJe4Z946/LiA39K/vy6XlxZ6Y+vnPf17jrzgkRnImHksR\n1vgBdDXKdhx9mR6bil1LU1yjYi9i/Iqxq9gXYcKECeFFSTOWcsgpNduIeWP2QQ04/crTy05zHqfS\nEveMe8cYBLE5znfffefLpRnEvpRRmOlilR/KNJPiL0RXXnll8p//+Z91jkpc39GHKy1lR4gudC1N\ndY2KvYjxK8auYl+E2CPOiBEjwsvXVD2JexZ7xqnUdvZtRez5sJvmVNn53A9qsmk2xXQc0Iz8uOGG\nG8IPUuZnRx/mQ3QGSos/BuJoxkzzPcmll14a/lLDdx98iM3H048++mjYvnPnzuEYueeXux37LHQu\n1LpnR4jmo/d47ELnRY19FPvstfDXingOufOzIztz/B07dij2IsaviGLfWGKPOLwcleXqStyzbM84\n9KzDXy18uTS92Nc1WBvCyncO1F7TheqMGTPCvNtvvz0dVCyOPEyNPlIcP9jOjj5Mk7FCoxcXGiEZ\nYee4jEQcR0HON+py3C5O5zuXQiNEFxtVOTsQW6GRlLPz2bbQyM51Dc6m2IsYv2LsKvYliD01aPQm\nk5vosrJ9+/ahVjLf8mx66aWXQneWxdbhBU9bd2SoPvtuylSu45aaSslHEvcsK/aV+AFtWxH73B6i\nSIxTQC0+6/BRN6JPzThdnMbuV+kZKTv6cLHRi3NHSM43nd0232jQdZ1LoRGii51X9v+FRlLOzi82\nsrNiL2L8irGr2DcCepIpJva8tM8444w6JZiXM21o6cKy2HosZ7RX/k9zgFL23dSp1GsqRyo1H7Ni\n361bt1RwJk6c6MulDGKPtPJjjOk41sEzzzyTrpPt6jSbuGdxe0S7LoFGyFk/LsvWdOduG1PudsXO\nhX1lR4jm+UCtfqlin72WfHnE/Oeffz40H8ouo1mOYi9i/Ioo9k0kLllhnDp1anLkkUeGRM8eiGOU\n4KVLl6YDQV133XXJ9u3bw/zf/va3af/stJel1pntmXf22WcnixYtCutRK/i73/0u7I+a8yjY9Czy\n2GOPpdfLoFD5xLvQfpELPlR8+OGH0z71OddSrimmYudAbXjsA582wfG6+ZGSzQ+awsTzoVaS5hEc\nk1rRQnmXTbn5WGj/MeWKDl1fFoOPbn251L98lDIKMwOnxXnUfHPdNJeJoxnH7lZJMXbYJjuKc7HR\ni3NHSM43HdfNHXU5d2TlQudSaITous4r/r/QiNTZ+cVGdq5rxGXFXsT4FWNXsS9BXJBaUmy3i5gs\nW7YsvMyRYF70mzZtSj+oo20s0kqTFmqZqaWkDS1iiWyz3pQpU8J6fDjHerzAaSqCzMemOPH/tElm\newRi3rx5YXtkI54XCTEutF/2xbIbb7wx/CUA+Y7nXeyasvsvdA6xTTLXTVvm8847L+w/HjPOv+qq\nq5ITTzwx5EdcRh7993//d9hnobzLnkM2H8mrQvuP6+eKDjX2+XrGYaTcLl26tPiDvq2KPZLLx6z8\nGIxN0MgH7iv3bcyYMWGaH280eYmjOMdmMcQf+4ujF+cKdD6RLyb22fnFzoVY5Vz4ixHXxTUg9vwg\nL+W8stfCdcdzyM6Pzwc+/uUYcWRnesxR7EWMXzF2FfsmFHuaD8TabKYRWwQUCaa/dF7OfAhIswM+\n9OPc+BiOFzfSyXwkYM6cOWF7esB45ZVX0n1kZT77f/ZNzfa0adOCONALR6701rVftqcWnOXZfRe7\npuz+C50D22fX5zwmT55ca780M2B7JDr3fIrlXfYcsvlYbP+FxD7b5eUPP/wQfgSdcMIJNZpc+HKp\nf/moaxTmKPa5ox1z7TR9yR15mOZWSG4cXC2OPozsFhq9OHeE5Lqm840AHecVOpdCI0QXG1U5u/9C\nIynnzi82snOxpNiLGL9i7Cr2JYgL8kgaPXp06B89TkdBpmYt+zLOJkSZP7EjpIgrcvrggw/WWAdB\nZR8Ibdxf9v9sQ7eNcf3LL788CHQ8D1Jd++X4rMO62X0Xu6bc/ec7B2o077jjjhrrkpj/0EMPpdPk\nwSmnnBLyg+Mj8sxjWbG8y+4zm4/F9h/n5RN7fpDQpOiII45Ij0Mba8W+ecS+PimOPIzoFht9uNjo\nxS1xLoWOX+p5FRpJOTu/2OjNir2I8SvGrmLfRGJPk484zaBOWUGmX25qKEm84OkeDwmNQsqf8+n1\n4uijjw7NXuLAULHJTCGxp19smrywH2r0mJ+VWlJd+43/zyf2ha4pu/9C5xCvGxFiPcSZ2kv6ke/V\nq1cN8Ubmo9hnz6dY3uUTe/KRbQrtP5/Y8+OBbwuyPxzOOeec8NeFcj3oFXtTUybFXsT4FWNXsa+H\n2Me+q2lLy5/g6WM61orH3j4QRWra6BObaT62oxkC0km7XeYjp4go4sqf81nGn/lp755P7BkUh3X4\neI/taMObK/aIbqH9ZveVK/bFrim7/0LnwA8BtqcPbuZTe8/2CHZ2/tChQ8MPD46Rez7F8i57Dtl8\npFlGof3niv1ZZ51VQ+h79uwZmvnkiyVT/RL525pGLK7mpNiLGL9i7Cr2JYg9tdExMRptVmy6du0a\naplZRld1cf5hhx0WBJT5CCwflDKfdY499th0vfvvvz+sS08xfNgaPxrN/p8a6l/96lfpNhdeeGEQ\n7Ox5Iaql7Jd1c6eLXVNMxc4he92cA+fC/PjhIYn27/yIyHf8YnmXTdl85HuCQvuPKd6/7LXRZnrg\nwIG1RqFV0hV7xV4xEuVIRLFvY2JPooaYmuHc+XEZH4UiofmWZUWZ2ul8+yiUqLEmFVunIfut65pK\nOYd43Q3db115Vygfi+0/3j8+JKaW/l/+5V9qSOktt9yS1tzbFKfh5cORjisjKfYixq8Yu4p9EejT\nnRclPa0gm6bqSdwz7l2PHj1SwaGP8MGDB9fqCYe/Pij2rVfs6U6VnmnoKYm/FtGlK021GrNPxozg\nB6ViL6IciSj2VVI4+/fvr9hXudhfe+21NfqxZxnQ1WXsu97uLlu32MdvOhB7/trE9x+N2R8xRJMu\n9qXYiyhHIop9lYn99OnTleUqS9yzYmIfoZ/xW2+9VbFvhNjnjhBczpRvFOTsKM7Z0Z2LjRZdbLTm\n7AjIlXTtir2I8SvGrmJfhAkTJoQXJaObxq4YTdWRuGfcu0GDBhUV+wgfHyL4vlyqV+zjCLCMEMsA\nWNlRkKPY0xQn/p9el+L6dOcaRzwmRuJoyIzWTI0/zfLigGgIPqPB8uNRsRdRjkQU+yoTe9plK8vV\nlbhnuWL/6quvpiPP+nJpOrHPjg5czlRoFORiIzoXGvG42GjN2RGQK+Xa8420rNiLGL9i7Cr2GbZt\n2xZelNTaxVElTdWRuGfcO+QtW2Ov2LdesR87dmwYRyF3fiGxz3axmk18n8F6yDs/EnP38cEHH4Rl\nUfoVexHlSESxr4LCyUudF+UFF1xQNUKLgND8APlgmlrIOE03mI888kiabwzqFNejD/hTTjklzKdd\n+qZNm9L90QSBpgjUYFZLPnDPuHeTJk1S7NuI2FNjT/MbmsswTQwzCjKjMecT+7h+/C6D8s6Ix7Fd\nfrb2P5/YW2OvGIlyJKLYV1nhjC9LPqZDjCs9MYrr6aefHvp3z52eOnVqGJ2VEW5nzpyZDvTEtfF/\nRpflY1Ikng8Qd+/enY4syzyWV0MecD35JIemOL5cmr5sZEcHLmcqNAryypUr05GOs6M7FxvxuNho\nzdkRkCvl2rMjLSv2IsavGLuKfQFizzjTpk2rCqmldjIr9tlpRmql1p0Bm2hmhAAj78xnxNiNGzcm\nW7ZsCduQrwgRssQ21H5Ww/WTuFe5PeKQEDhfLq1X7EkvvPBCmq/INz9UkfL4UWz2/7nr04UlH8RG\nkc+ul52m1j6OgKzYiyhHIop9FRXO+AEt3d5Vu9gj8bGrPtJll10WxIe2yfnaGtMsge2RfpodVIvY\nc69yP5ylG0R+qPhyaXqxr+8ox82dCo2C3NgRj4uNgFwJSbEXMX7F2FXs64AeMnhZ0qc1bWsrPfEi\nP/nkk4PEM7148eIg9nv27AldANKNH5JOcwXm/+EPf0hGjBgRRuZkGxJNDWbMmBFq8BF7rp/tq+H6\nSdwrzpmmFVFuaHrEXyN8ubR+sW+rSbEXMX7F2FXs6+Cnn35KX5hbt26teKmlVpo8oTkKfXMj7Bdd\ndFEQcz6Wpfad+Tt27AgfDiL29ALCNnxounPnztCvN9N0Fxhr/KtF7LlH8X7NmzcvlZuXX345jM7p\ny6Xpxd4B0SojKfYixq8Yu4p9CfDhKC/MP/3pT+HP75WcEPB77703zRtGzOzSpUuoieejQJrfxGX0\nHIPks122Oc5hhx0W2qMzn8F4uHa2r/RrJ3GPON8ePXrUkBvEvpJ6xFHsTYq9iM9eEcW+DIWTZim8\nMOlloxrklsTHsaT6LKOJDh/J8uOgWq4zN3GP4qBildwjjmJvUuxFfPaKKPZlKJxff/11+tKk1hvx\nNVVe4t7E+0RXhJX84WxrEntHOq6MpNiLGL9i7Cr2JRKb44wfP16JrtDEvcnXDAfJr7T29Yq9SbEX\n8dkrotiXqXDG5jj0uKJEV2aKveHka4bz448/+nJpJrGvltGIW3tS7EWMXzF2FfsSyfaOw4iVsWtI\nU2Uk7km+3nBIjK7ry0WxV+yNXRHjV4xdxT6F7iJ5cQ4cOFCZrrDEPeHe/OY3v6nVDId++X25KPaK\nvbErYvyKsavYp2TbsTLQE8JoKn/iXsT7Ql/81dAMpzWJfbWMSNzak2IvYvyKsavY15P+/fuHlyd9\nxSvVlZG4F9yTa6+9tobQLFy4MFmwYIEvF8VesTd2RYxfMXYV+9rQ/7u19pVZWz9u3LhatfWHDh3y\n5dLMYl8NIxK3haTYixi/Yuwq9g0g1tr369cv2blzp6mMiXuQr7aeNH36dF8uir1ib+yKGL9i7Cr2\nhckOWDV79mwFu0yJvM83IFX8aJbmCb5cFHvF3tgVMX7F2FXsizJq1KjwEu3evbuSXaZE3nMP7r77\n7loyg/T/85//9OXSAmK/d+9eUwUkxV7E+BVjV7FvIPRrf+GFF4YX6bBhwxTtFk7kOXnfqVOnWv3W\nU1tPF5i+XBR7xV6xFzF+xdhV7Eti8eLF6cuU/yvcLZOy+T58+PC8besrvba+NYm9ox5XRlLsRYxf\nMXYV+0YSP6SlWcjGjRsV72ZO5HFsgpPvg1lq6w8ePOjLRbFX7BV7EeNXjF3Fvn5km+TYS07L9YKT\nrwkO/dYvW7bMl4tir9gr9iLGrxi7in3DyPZtP3LkSAW8mRJ5W6jPetLLL7+cfPfdd75cWljs+Z7B\nVP6k2IsYv2LsKvZNxIQJE+wCs4W6thw0aFAteZk5c2ZVfDCr2JsUexGfvSKKfRUUToSTF+v555/v\nx7RN/LEseUre9u3bt5a40ARn0aJFVfHBbGsUe0dAroyk2IsYv2LsKvZNCO3te/fu7ce0zfSxbI8e\nPWq1qydNnTo1+fHHHy2gir1ir9iLGL9i7Cr2TSv38WPaq6++WrlvpNSTh4U+lo294Hz22WcW0DKK\nvbFaGUmxFzF+xdhV7JuB/fv3K/dNLPWTJk3K266e9Sygir1JsRcxfsXYVexbRO5tc9/wNvWFpH7u\n3LmhXb0FtPxib6qspNiLGL9i7Cr2zST3sc29cl9/qadNfT6p52NZxL7aPpZV7E2KvYjPXhHFvooL\nJ23ue/Xqpdw3QOrztalH6qv1Y9m28HIZOnRorXtmKm8ydkWMXzF2FftmlHv7uc/fT31bknrF3qTY\ni/jsFVHsq7RwZuWe9MILLyj0/5/Ii5gvhaSepjfTp09vNVLfWl8uEydONFVQUuxFjF8xdhX7ZpT7\nIUOGpBLbr1+/Nt1jDtdOHsT8+M1vflNQ6qtxACpfLuLLRcT4FVHsW3nhfPrpp0NvL3Egq7bY7p5r\njgNPkRd333133mYErVXqfbmIsSti/Ioo9q2kcO7duzfp2rVrWls9cuTINlF7zzVyrfG6u3Xrlrfn\nm9Yu9b5cxNgVMX5FFPtWVDi///775Lrrrkslt7XX3mdr6UnXXntt3qY3pJdffjlZuXJlq5V6Xy5i\n7IoYvyKKfSssnGPHjk06duyYCu+9996brFmzptUIPdfCNcXro+nN8OHD8wo9Pd+MHz++qkeU9eUi\nxq6I8Sui2Lfhwvnxxx+Hj0ej/NL1IzJczc1zOHeuIXZjSaItfaFa+vnz54fuLL/99lsLqIixK2L8\niij21cs//vGP0EXdlVdeWUvwq6kGP9bQZ4WeZjeF2tLHpje0pycPLKAixq6I8Sui2Fc9tCmnpnvY\nsGHJ5ZdfXmOo+D59+lR0//ecG+eYPWf6pR83blxBoecDWaSea27N7el9uYixK2L8iij2bbRwMhAT\n0vvkk0/WaKITP7J98MEHK2IEW86Bc8l+FBv7pC8m9KRXX301XGNrGnTK+BVjV8T4FVHsLZx5OXjw\nYDJ58uRkxIgRyaBBg0L3kFmBprkLgzxRW94SzXU4BsfimNmmNrHrSs6xWJOb2OyGa+LaLKC+XMTY\nFTF+RRT7NlM4aaLy3nvvBWGmlhvJp0Y8V/Kj6NMchlp0+ounRr0hws82bMs+2Bf7zBX5KPOcC+sV\nk3l6u+HcuQaupa01u/HlIsauiPErothbOFP4sHTHjh1BjvnIll5k+P/gwYPDx6n5RL+pE8fgWByz\nrpr52NMN58q6nHtb+jjW+BVjV8T4FVHsLZxFobabZiwzZ84M7dipCY8ijUBTe06TmL59+wYJp9/4\n+go827At+2Bf7LMUkc+2n+fcOEfO1Rp641eMXRHjV0Sxt3AW4bvvvku2bNkSBnWi7TpCTS15qQLe\nVIljcmzOgXPhnDg3MX7F2BUxfkUUewtnPWFgJ4R6zpw5ofkLCdGm1rypRZ59su94HI7JsdvK4FLG\nrxi7xq4YvyKKvYWzRaALSSSbD1VXrlwZ5Jua9CjiDU1xH+yTfXOMttpdpfErxq6I8Sui2Fs4ywIf\nrtI85tChQ8mBAwfqldiGbf341fgVMXbF+BVR7C2cIsavGLsixq+IYm/hFAuoiLErYvyKsavYi/hy\nETF2xfg1fsXYVexFjF8RY1eMXxHF3sIpYvyKsSti/Ioo9hZOsYCKGLsixq8Yu4q9iC8XEWNXxPgV\nY1exFzF+RYxdMX5FFHsLp4jxK8auiPErothbOMUCKmLsihi/Ioq9iC8XEWNXxPgVY1exFzF+RYxd\nMX5FFPsWvECTyWQymUwmk6ktJMXeZDKZTCaTyWRS7CtX7EVERERERLEXEREREVHsRUREREREsRcR\nEREREcVeREREREQUexERERERxV5ERERERBR7ERERERFR7EVERERERLEXERERERHFXkREREREsRcR\nEREREcVeREREREQUexERERERUexFRERERBR7ERERERFR7EVERERERLEXERERERHFXkRERESk9fF/\nYPl9GgY8LikAAAAASUVORK5CYII=\n",
       "prompt_number": 1,
       "text": [
        "<IPython.core.display.Image object at 0x10a1cd250>"
       ]
      }
     ],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## About this tutorial\n",
      "\n",
      "### References\n",
      "We've put this together from our experience and a number of sources, please check the references at the bottom of this document.\n",
      "\n",
      "### Dataset\n",
      "MovieLens from GroupLens Research: [grouplens.org](http://www.grouplens.org/)\n",
      "\n",
      "The MovieLens 1M data set contains 1 million ratings collected from 6000 users\n",
      "on 4000 movies.\n",
      "\n",
      "### What this tutorial is\n",
      "\n",
      "The goal of this tutorial is to provide you with a hands-on overview of two of\n",
      "the main libraries from the scientific and data analysis communities. We're going to\n",
      "use:\n",
      "\n",
      "- numpy -- [numpy.org](http://www.numpy.org)\n",
      "- pandas -- [pandas.pydata.org](http://pandas.pydata.org/)\n",
      "- (bonus) pytables -- [pytables.org](http://www.pytables.org)\n",
      "\n",
      "### What this tutorial is not\n",
      "\n",
      "- An exhaustive overview of the recommendation literature\n",
      "- A set of recipes that will win you the next Netflix/Kaggle/etc challenge."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Roadmap\n",
      "\n",
      "What exactly are we going to do? Here's a high-level overview:\n",
      "\n",
      "- learn about NumPy arrays\n",
      "- learn about Series and DataFrames\n",
      "- iterate over a few implementations of a minimal reco engine\n",
      "- challenge"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## NumPy: Numerical Python\n",
      "\n",
      "### What is it?\n",
      "\n",
      "*It is a Python library that provides a multidimensional array object, various\n",
      "derived objects (such as masked arrays and matrices), and an assortment of\n",
      "routines for fast operations on arrays, including mathematical, logical, shape\n",
      "manipulation, sorting, selecting, I/O, discrete Fourier transforms, basic linear\n",
      "algebra, basic statistical operations, random simulation and much more.*"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np\n",
      "\n",
      "# set some print options\n",
      "np.set_printoptions(precision=4)\n",
      "np.set_printoptions(threshold=5)\n",
      "np.set_printoptions(suppress=True)\n",
      "\n",
      "# init random gen\n",
      "np.random.seed(2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### NumPy's basic data structure: the ndarray\n",
      "\n",
      "Think of ndarrays as the building blocks for pydata. A multidimensional array\n",
      "object that acts as a container for data to be passed between algorithms. Also,\n",
      "libraries written in a lower-level language, such as C or Fortran, can operate\n",
      "on the data stored in a NumPy array without copying any data.\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np\n",
      "\n",
      "# build an array using the array function\n",
      "arr = np.array([0, 9, 5, 4, 3])\n",
      "arr"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 3,
       "text": [
        "array([0, 9, 5, 4, 3])"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Array creation examples\n",
      "\n",
      "There are several functions that are used to create new arrays:\n",
      "\n",
      "- `np.array`\n",
      "- `np.asarray`\n",
      "- `np.arange`\n",
      "- `np.ones`\n",
      "- `np.ones_like`\n",
      "- `np.zeros`\n",
      "- `np.zeros_like`\n",
      "- `np.empty`\n",
      "- `np.random.randn` and other funcs from the random module"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.zeros(4)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 4,
       "text": [
        "array([ 0.,  0.,  0.,  0.])"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.ones(4)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 5,
       "text": [
        "array([ 1.,  1.,  1.,  1.])"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.empty(4)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 6,
       "text": [
        "array([ -2.3158e+077,  -2.3158e+077,   6.9467e-310,   2.8248e-309])"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.arange(4)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 7,
       "text": [
        "array([0, 1, 2, 3])"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### dtype and shape\n",
      "\n",
      "NumPy's arrays are containers of homogeneous data, which means all elements are\n",
      "of the same type. The 'dtype' propery is an object that specifies the data type\n",
      "of each element. The 'shape' property is a tuple that indicates the size of each\n",
      "dimension."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr = np.random.randn(5)\n",
      "arr"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 8,
       "text": [
        "array([-0.4168, -0.0563, -2.1362,  1.6403, -1.7934])"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr.dtype"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 9,
       "text": [
        "dtype('float64')"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "(5,)"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# you can be explicit about the data type that you want\n",
      "np.empty(4, dtype=np.int32)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "array([         0, -805306368,          0, -805306368], dtype=int32)"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.array(['numpy','pandas','pytables'], dtype=np.string_)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 12,
       "text": [
        "array(['numpy', 'pandas', 'pytables'], \n",
        "      dtype='|S8')"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "float_arr = np.array([4.4, 5.52425, -0.1234, 98.1], dtype=np.float64)\n",
      "# truncate the decimal part\n",
      "float_arr.astype(np.int32)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 13,
       "text": [
        "array([ 4,  5,  0, 98], dtype=int32)"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Indexing and slicing\n",
      "\n",
      "#### Just what you would expect from Python"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr = np.array([0, 9, 1, 4, 64])\n",
      "arr[3]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 14,
       "text": [
        "4"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr[1:3]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 15,
       "text": [
        "array([9, 1])"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr[1:4:2]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 16,
       "text": [
        "array([9, 4])"
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr[::2]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 17,
       "text": [
        "array([ 0,  1, 64])"
       ]
      }
     ],
     "prompt_number": 17
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr[:2]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 18,
       "text": [
        "array([0, 9])"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr[-2:]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 19,
       "text": [
        "array([ 4, 64])"
       ]
      }
     ],
     "prompt_number": 19
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# set the last two elements to 555\n",
      "arr[-2:] = 555\n",
      "arr"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 20,
       "text": [
        "array([  0,   9,   1, 555, 555])"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### (BONUS) Indexing behaviour for multidimensional arrays\n",
      "\n",
      "A good way to think about indexing in multidimensional arrays is that you are\n",
      "moving along the values of the shape property. So, a 4d array `arr_4d`, with a\n",
      "shape of `(w,x,y,z)` will result in indexed views such that:\n",
      "\n",
      "- `arr_4d[i].shape == (x,y,z)`\n",
      "- `arr_4d[i,j].shape == (y,z)`\n",
      "- `arr_4d[i,j,k].shape == (z,)`\n",
      "\n",
      "For the case of slices, what you are doing is selecting a range of elements\n",
      "along a particular axis:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "    arr_2d = np.array([[5,3,4],[0,1,2],[1,1,10],[0,0,0.1]])\n",
      "arr_2d"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 21,
       "text": [
        "array([[  5. ,   3. ,   4. ],\n",
        "       [  0. ,   1. ,   2. ],\n",
        "       [  1. ,   1. ,  10. ],\n",
        "       [  0. ,   0. ,   0.1]])"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# get the first row\n",
      "arr_2d[0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 22,
       "text": [
        "array([ 5.,  3.,  4.])"
       ]
      }
     ],
     "prompt_number": 22
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# get the first column\n",
      "arr_2d[:,0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 23,
       "text": [
        "array([ 5.,  0.,  1.,  0.])"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# get the first two rows\n",
      "arr_2d[:2]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 24,
       "text": [
        "array([[ 5.,  3.,  4.],\n",
        "       [ 0.,  1.,  2.]])"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### Careful, it's a view!\n",
      "\n",
      "A slice does not return a copy, which means that any modifications will be\n",
      "reflected in the source array. This is a design feature of NumPy to avoid memory\n",
      "problems."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr = np.array([0, 3, 1, 4, 64])\n",
      "arr"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 25,
       "text": [
        "array([ 0,  3,  1,  4, 64])"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "subarr = arr[2:4]\n",
      "subarr[1] = 99\n",
      "arr"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 26,
       "text": [
        "array([ 0,  3,  1, 99, 64])"
       ]
      }
     ],
     "prompt_number": 26
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### (Fancy) Boolean indexing\n",
      "\n",
      "Boolean indexing allows you to select data subsets of an array that satisfy a\n",
      "given condition."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr = np.array([10, 20])\n",
      "idx = np.array([True, False])\n",
      "arr[idx]\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 27,
       "text": [
        "array([10])"
       ]
      }
     ],
     "prompt_number": 27
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr_2d = np.random.randn(5)\n",
      "arr_2d\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 28,
       "text": [
        "array([-0.8417,  0.5029, -1.2453, -1.058 , -0.909 ])"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr_2d < 0"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 29,
       "text": [
        "array([ True, False,  True,  True,  True], dtype=bool)"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr_2d[arr_2d < 0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 30,
       "text": [
        "array([-0.8417, -1.2453, -1.058 , -0.909 ])"
       ]
      }
     ],
     "prompt_number": 30
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr_2d[(arr_2d > -0.5) & (arr_2d < 0)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 31,
       "text": [
        "array([], dtype=float64)"
       ]
      }
     ],
     "prompt_number": 31
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr_2d[arr_2d < 0] = 0\n",
      "arr_2d"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 32,
       "text": [
        "array([ 0.    ,  0.5029,  0.    ,  0.    ,  0.    ])"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### (Fancy) list-of-locations indexing\n",
      "\n",
      "Fancy indexing is indexing with integer arrays."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr = np.array([100, 101, 130, 131, 321, 123])\n",
      "arr[[1, 3, 4]]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 33,
       "text": [
        "array([101, 131, 321])"
       ]
      }
     ],
     "prompt_number": 33
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr = np.arange(18).reshape(6,3)\n",
      "arr"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 34,
       "text": [
        "array([[ 0,  1,  2],\n",
        "       [ 3,  4,  5],\n",
        "       [ 6,  7,  8],\n",
        "       [ 9, 10, 11],\n",
        "       [12, 13, 14],\n",
        "       [15, 16, 17]])"
       ]
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# fancy selection of rows in a particular order\n",
      "arr[[0,4,4]]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 35,
       "text": [
        "array([[ 0,  1,  2],\n",
        "       [12, 13, 14],\n",
        "       [12, 13, 14]])"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# index into individual elements and flatten\n",
      "arr[[5,3,1],[2,1,0]]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 36,
       "text": [
        "array([17, 10,  3])"
       ]
      }
     ],
     "prompt_number": 36
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# select a submatrix\n",
      "arr[np.ix_([5,3,1],[2,1])]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 37,
       "text": [
        "array([[17, 16],\n",
        "       [11, 10],\n",
        "       [ 5,  4]])"
       ]
      }
     ],
     "prompt_number": 37
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "--> Go to \"Numpy Questions: Indexing\""
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Vectorization\n",
      "Vectorization is at the heart of NumPy and it enables us to express operations\n",
      "without writing any for loops.  Operations between arrays with equal shapes are\n",
      "performed element-wise.\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr1 = np.array([0, 9, 1.02, 4, 32])\n",
      "arr2 = np.array([1, 45, 7.8, 5, 90])\n",
      "arr1 - arr2"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 38,
       "text": [
        "array([ -1.  , -36.  ,  -6.78,  -1.  , -58.  ])"
       ]
      }
     ],
     "prompt_number": 38
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr1 * arr1"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 39,
       "text": [
        "array([    0.    ,    81.    ,     1.0404,    16.    ,  1024.    ])"
       ]
      }
     ],
     "prompt_number": 39
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Broadcasting Rules\n",
      "\n",
      "Vectorized operations between arrays of different sizes and between arrays and\n",
      "scalars are subject to the rules of broadcasting. The idea is quite simple in\n",
      "many cases:\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr = np.array([0, 9, 1.02, 4, 64])\n",
      "5 * arr "
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 40,
       "text": [
        "array([   0. ,   45. ,    5.1,   20. ,  320. ])"
       ]
      }
     ],
     "prompt_number": 40
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "10 + arr"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 41,
       "text": [
        "array([ 10.  ,  19.  ,  11.02,  14.  ,  74.  ])"
       ]
      }
     ],
     "prompt_number": 41
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "arr ** .5"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 42,
       "text": [
        "array([ 0.  ,  3.  ,  1.01,  2.  ,  8.  ])"
       ]
      }
     ],
     "prompt_number": 42
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The case of arrays of different shapes is slightly more complicated. The gist of it is that the shape of the operands need to conform to a certain specification. Don't worry if this does not make sense right away."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "mtx = np.random.randn(4,2)\n",
      "mtx"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 43,
       "text": [
        "array([[ 0.5515,  2.2922],\n",
        "       [ 0.0415, -1.1179],\n",
        "       [ 0.5391, -0.5962],\n",
        "       [-0.0191,  1.175 ]])"
       ]
      }
     ],
     "prompt_number": 43
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vec = np.array([100, 100])\n",
      "vec"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 44,
       "text": [
        "array([100, 100])"
       ]
      }
     ],
     "prompt_number": 44
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "mtx + vec"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 45,
       "text": [
        "array([[ 100.5515,  102.2922],\n",
        "       [ 100.0415,   98.8821],\n",
        "       [ 100.5391,   99.4038],\n",
        "       [  99.9809,  101.175 ]])"
       ]
      }
     ],
     "prompt_number": 45
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "mean_row = np.mean(mtx, axis=0)\n",
      "mean_row"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 46,
       "text": [
        "array([ 0.2782,  0.4383])"
       ]
      }
     ],
     "prompt_number": 46
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "centered_rows = mtx - mean_row\n",
      "centered_rows"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 47,
       "text": [
        "array([[ 0.2732,  1.8539],\n",
        "       [-0.2367, -1.5562],\n",
        "       [ 0.2608, -1.0344],\n",
        "       [-0.2974,  0.7367]])"
       ]
      }
     ],
     "prompt_number": 47
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.mean(centered_rows, axis=0)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 48,
       "text": [
        "array([-0.,  0.])"
       ]
      }
     ],
     "prompt_number": 48
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "mean_col = np.mean(mtx, axis=1)\n",
      "mean_col"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 49,
       "text": [
        "array([ 1.4218, -0.5382, -0.0286,  0.5779])"
       ]
      }
     ],
     "prompt_number": 49
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "centered_cols = mtx - mean_col"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "ValueError",
       "evalue": "operands could not be broadcast together with shapes (4,2) (4,) ",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
        "\u001b[0;32m<ipython-input-50-26322f66ff99>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mcentered_cols\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmtx\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mmean_col\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
        "\u001b[0;31mValueError\u001b[0m: operands could not be broadcast together with shapes (4,2) (4,) "
       ]
      }
     ],
     "prompt_number": 50
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# make the 1-D array a column vector\n",
      "mean_col.reshape((4,1))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 51,
       "text": [
        "array([[ 1.4218],\n",
        "       [-0.5382],\n",
        "       [-0.0286],\n",
        "       [ 0.5779]])"
       ]
      }
     ],
     "prompt_number": 51
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "centered_cols = mtx - mean_col.reshape((4,1))\n",
      "centered_rows"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 52,
       "text": [
        "array([[ 0.2732,  1.8539],\n",
        "       [-0.2367, -1.5562],\n",
        "       [ 0.2608, -1.0344],\n",
        "       [-0.2974,  0.7367]])"
       ]
      }
     ],
     "prompt_number": 52
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "centered_cols.mean(axis=1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 53,
       "text": [
        "array([-0.,  0.,  0., -0.])"
       ]
      }
     ],
     "prompt_number": 53
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### A note about NANs: \n",
      "\n",
      "Per the floating point standard IEEE 754, NaN is a floating point value that, by definition, is not equal to any other floating point value."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.nan != np.nan"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 54,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 54
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.array([10, 5, 4, np.nan, 1, np.nan]) == np.nan"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 55,
       "text": [
        "array([False, False, False, False, False, False], dtype=bool)"
       ]
      }
     ],
     "prompt_number": 55
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.isnan(np.array([10, 5, 4, np.nan, 1, np.nan]))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 56,
       "text": [
        "array([False, False, False,  True, False,  True], dtype=bool)"
       ]
      }
     ],
     "prompt_number": 56
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "--> Go to \"Numpy Questions: Operations\""
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## pandas: Python Data Analysis Library\n",
      "\n",
      "### What is it?\n",
      "\n",
      "*Python has long been great for data munging and preparation, but less so for\n",
      "data analysis and modeling. pandas helps fill this gap, enabling you to carry\n",
      "out your entire data analysis workflow in Python without having to switch to a\n",
      "more domain specific language like R.*\n",
      "\n",
      "The heart of pandas is the DataFrame object for data manipulation. It features:\n",
      "\n",
      "- a powerful index object\n",
      "- data alignment\n",
      "- handling of missing data\n",
      "- aggregation with groupby\n",
      "- data manipuation via reshape, pivot, slice, merge, join"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import pandas as pd\n",
      "\n",
      "pd.set_option('precision', 3, 'notebook_repr_html', True, )"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 57
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Series: labelled arrays\n",
      "\n",
      "The pandas Series is kind of like an ndarray (used to actually be a subclass of it) that supports more meaninful indices."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### Let's look at some creation examples for Series"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "values = np.array([2.0, 1.0, 5.0, 0.97, 3.0, 10.0, 0.0599, 8.0])\n",
      "ser = pd.Series(values)\n",
      "ser"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 58,
       "text": [
        "0     2.00\n",
        "1     1.00\n",
        "2     5.00\n",
        "3     0.97\n",
        "4     3.00\n",
        "5    10.00\n",
        "6     0.06\n",
        "7     8.00\n",
        "dtype: float64"
       ]
      }
     ],
     "prompt_number": 58
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "values = np.array([2.0, 1.0, 5.0, 0.97, 3.0, 10.0, 0.0599, 8.0])\n",
      "labels = ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H']\n",
      "ser = pd.Series(data=values, index=labels)\n",
      "ser\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 59,
       "text": [
        "A     2.00\n",
        "B     1.00\n",
        "C     5.00\n",
        "D     0.97\n",
        "E     3.00\n",
        "F    10.00\n",
        "G     0.06\n",
        "H     8.00\n",
        "dtype: float64"
       ]
      }
     ],
     "prompt_number": 59
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "movie_rating = {\n",
      "    'age': 1,\n",
      "    'gender': 'F',\n",
      "    'genres': 'Drama',\n",
      "    'movie_id': 1193,\n",
      "    'occupation': 10,\n",
      "    'rating': 5,\n",
      "    'timestamp': 978300760,\n",
      "    'title': \"One Flew Over the Cuckoo's Nest (1975)\",\n",
      "    'user_id': 1,\n",
      "    'zip': '48067'\n",
      "    }\n",
      "ser = pd.Series(movie_rating)\n",
      "print ser\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "age                                                1\n",
        "gender                                             F\n",
        "genres                                         Drama\n",
        "movie_id                                        1193\n",
        "occupation                                        10\n",
        "rating                                             5\n",
        "timestamp                                  978300760\n",
        "title         One Flew Over the Cuckoo's Nest (1975)\n",
        "user_id                                            1\n",
        "zip                                            48067\n",
        "dtype: object\n"
       ]
      }
     ],
     "prompt_number": 60
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser.index"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 61,
       "text": [
        "Index([u'age', u'gender', u'genres', u'movie_id', u'occupation', u'rating', u'timestamp', u'title', u'user_id', u'zip'], dtype='object')"
       ]
      }
     ],
     "prompt_number": 61
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser.values"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 62,
       "text": [
        "array([1, 'F', 'Drama', ..., \"One Flew Over the Cuckoo's Nest (1975)\", 1,\n",
        "       '48067'], dtype=object)"
       ]
      }
     ],
     "prompt_number": 62
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### Series indexing"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser.loc['gender']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 63,
       "text": [
        "'F'"
       ]
      }
     ],
     "prompt_number": 63
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser.loc[['gender', 'zip']]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 64,
       "text": [
        "gender        F\n",
        "zip       48067\n",
        "dtype: object"
       ]
      }
     ],
     "prompt_number": 64
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "bool_arr = np.array([False, False, False, False, False, True, False, False, False, False])\n",
      "bool_arr"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 65,
       "text": [
        "array([False, False, False, ..., False, False, False], dtype=bool)"
       ]
      }
     ],
     "prompt_number": 65
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser.loc[bool_arr]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 66,
       "text": [
        "rating    5\n",
        "dtype: object"
       ]
      }
     ],
     "prompt_number": 66
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser.iloc[1]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 67,
       "text": [
        "'F'"
       ]
      }
     ],
     "prompt_number": 67
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser.iloc[[1,2]]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 68,
       "text": [
        "gender        F\n",
        "genres    Drama\n",
        "dtype: object"
       ]
      }
     ],
     "prompt_number": 68
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser.ix['gender']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 69,
       "text": [
        "'F'"
       ]
      }
     ],
     "prompt_number": 69
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser.ix[1]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 70,
       "text": [
        "'F'"
       ]
      }
     ],
     "prompt_number": 70
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser['gender']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 71,
       "text": [
        "'F'"
       ]
      }
     ],
     "prompt_number": 71
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser[[1,2]]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 72,
       "text": [
        "gender        F\n",
        "genres    Drama\n",
        "dtype: object"
       ]
      }
     ],
     "prompt_number": 72
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### Operations between Series with different index objects"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser_1 = pd.Series(data=[1,3,4], index=['A', 'B', 'C'])\n",
      "ser_2 = pd.Series(data=[5,5,5], index=['A', 'G', 'C'])\n",
      "print ser_1 + ser_2\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "A     6\n",
        "B   NaN\n",
        "C     9\n",
        "G   NaN\n",
        "dtype: float64\n"
       ]
      }
     ],
     "prompt_number": 73
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Automatic upcasting when performing operations between Series with different dtypes:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "ser_1 = pd.Series(data=[1,3,4], index=['A', 'B', 'C'], dtype=np.int32)\n",
      "ser_2 = pd.Series(data=[5,5,5], index=['A', 'G', 'C'], dtype=np.float64)\n",
      "ser_1 + ser_2\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 74,
       "text": [
        "A     6\n",
        "B   NaN\n",
        "C     9\n",
        "G   NaN\n",
        "dtype: float64"
       ]
      }
     ],
     "prompt_number": 74
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### DataFrame\n",
      "\n",
      "The DataFrame is the 2-dimensional version of a Series.\n",
      "\n",
      "#### Let's look at some creation examples for DataFrame\n",
      "\n",
      "You can think of it as a spreadsheet whose columns are Series objects."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# build from a dict of equal-length lists or ndarrays\n",
      "pd.DataFrame({'col_1': [0.12, 7, 45, 10], 'col_2': [0.9, 9, 34, 11]})"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>col_1</th>\n",
        "      <th>col_2</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>0</th>\n",
        "      <td>  0.12</td>\n",
        "      <td>  0.9</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
        "      <td>  7.00</td>\n",
        "      <td>  9.0</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
        "      <td> 45.00</td>\n",
        "      <td> 34.0</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>3</th>\n",
        "      <td> 10.00</td>\n",
        "      <td> 11.0</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>4 rows \u00d7 2 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 75,
       "text": [
        "   col_1  col_2\n",
        "0   0.12    0.9\n",
        "1   7.00    9.0\n",
        "2  45.00   34.0\n",
        "3  10.00   11.0\n",
        "\n",
        "[4 rows x 2 columns]"
       ]
      }
     ],
     "prompt_number": 75
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can explicitly set the column names and index values as well."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "pd.DataFrame(data={'col_1': [0.12, 7, 45, 10], 'col_2': [0.9, 9, 34, 11]},\n",
      "             columns=['col_1', 'col_2', 'col_3'])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>col_1</th>\n",
        "      <th>col_2</th>\n",
        "      <th>col_3</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>0</th>\n",
        "      <td>  0.12</td>\n",
        "      <td>  0.9</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
        "      <td>  7.00</td>\n",
        "      <td>  9.0</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
        "      <td> 45.00</td>\n",
        "      <td> 34.0</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>3</th>\n",
        "      <td> 10.00</td>\n",
        "      <td> 11.0</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>4 rows \u00d7 3 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 76,
       "text": [
        "   col_1  col_2 col_3\n",
        "0   0.12    0.9   NaN\n",
        "1   7.00    9.0   NaN\n",
        "2  45.00   34.0   NaN\n",
        "3  10.00   11.0   NaN\n",
        "\n",
        "[4 rows x 3 columns]"
       ]
      }
     ],
     "prompt_number": 76
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "pd.DataFrame(data={'col_1': [0.12, 7, 45, 10], 'col_2': [0.9, 9, 34, 11]},\n",
      "             columns=['col_1', 'col_2', 'col_3'],\n",
      "             index=['obs1', 'obs2', 'obs3', 'obs4'])\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>col_1</th>\n",
        "      <th>col_2</th>\n",
        "      <th>col_3</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>obs1</th>\n",
        "      <td>  0.12</td>\n",
        "      <td>  0.9</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs2</th>\n",
        "      <td>  7.00</td>\n",
        "      <td>  9.0</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs3</th>\n",
        "      <td> 45.00</td>\n",
        "      <td> 34.0</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs4</th>\n",
        "      <td> 10.00</td>\n",
        "      <td> 11.0</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>4 rows \u00d7 3 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 77,
       "text": [
        "      col_1  col_2 col_3\n",
        "obs1   0.12    0.9   NaN\n",
        "obs2   7.00    9.0   NaN\n",
        "obs3  45.00   34.0   NaN\n",
        "obs4  10.00   11.0   NaN\n",
        "\n",
        "[4 rows x 3 columns]"
       ]
      }
     ],
     "prompt_number": 77
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can also think of it as a dictionary of Series objects."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "movie_rating = {\n",
      "    'gender': 'F',\n",
      "    'genres': 'Drama',\n",
      "    'movie_id': 1193,\n",
      "    'rating': 5,\n",
      "    'timestamp': 978300760,\n",
      "    'user_id': 1,\n",
      "    }\n",
      "ser_1 = pd.Series(movie_rating)\n",
      "ser_2 = pd.Series(movie_rating)\n",
      "df = pd.DataFrame({'r_1': ser_1, 'r_2': ser_2})\n",
      "df.columns.name = 'rating_events'\n",
      "df.index.name = 'rating_data'\n",
      "df"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th>rating_events</th>\n",
        "      <th>r_1</th>\n",
        "      <th>r_2</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating_data</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>gender</th>\n",
        "      <td>         F</td>\n",
        "      <td>         F</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>genres</th>\n",
        "      <td>     Drama</td>\n",
        "      <td>     Drama</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>movie_id</th>\n",
        "      <td>      1193</td>\n",
        "      <td>      1193</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating</th>\n",
        "      <td>         5</td>\n",
        "      <td>         5</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>timestamp</th>\n",
        "      <td> 978300760</td>\n",
        "      <td> 978300760</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>user_id</th>\n",
        "      <td>         1</td>\n",
        "      <td>         1</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>6 rows \u00d7 2 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 78,
       "text": [
        "rating_events        r_1        r_2\n",
        "rating_data                        \n",
        "gender                 F          F\n",
        "genres             Drama      Drama\n",
        "movie_id            1193       1193\n",
        "rating                 5          5\n",
        "timestamp      978300760  978300760\n",
        "user_id                1          1\n",
        "\n",
        "[6 rows x 2 columns]"
       ]
      }
     ],
     "prompt_number": 78
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df = df.T\n",
      "df"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th>rating_data</th>\n",
        "      <th>gender</th>\n",
        "      <th>genres</th>\n",
        "      <th>movie_id</th>\n",
        "      <th>rating</th>\n",
        "      <th>timestamp</th>\n",
        "      <th>user_id</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating_events</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>r_1</th>\n",
        "      <td> F</td>\n",
        "      <td> Drama</td>\n",
        "      <td> 1193</td>\n",
        "      <td> 5</td>\n",
        "      <td> 978300760</td>\n",
        "      <td> 1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>r_2</th>\n",
        "      <td> F</td>\n",
        "      <td> Drama</td>\n",
        "      <td> 1193</td>\n",
        "      <td> 5</td>\n",
        "      <td> 978300760</td>\n",
        "      <td> 1</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>2 rows \u00d7 6 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 79,
       "text": [
        "rating_data   gender genres movie_id rating  timestamp user_id\n",
        "rating_events                                                 \n",
        "r_1                F  Drama     1193      5  978300760       1\n",
        "r_2                F  Drama     1193      5  978300760       1\n",
        "\n",
        "[2 rows x 6 columns]"
       ]
      }
     ],
     "prompt_number": 79
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.columns "
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 80,
       "text": [
        "Index([u'gender', u'genres', u'movie_id', u'rating', u'timestamp', u'user_id'], dtype='object')"
       ]
      }
     ],
     "prompt_number": 80
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.index"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 81,
       "text": [
        "Index([u'r_1', u'r_2'], dtype='object')"
       ]
      }
     ],
     "prompt_number": 81
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.values"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 82,
       "text": [
        "array([['F', 'Drama', 1193, 5, 978300760, 1],\n",
        "       ['F', 'Drama', 1193, 5, 978300760, 1]], dtype=object)"
       ]
      }
     ],
     "prompt_number": 82
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### Adding/Deleting entries"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df = pd.DataFrame({'r_1': ser_1, 'r_2': ser_2})\n",
      "df.drop('genres', axis=0)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>r_1</th>\n",
        "      <th>r_2</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating_data</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>gender</th>\n",
        "      <td>         F</td>\n",
        "      <td>         F</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>movie_id</th>\n",
        "      <td>      1193</td>\n",
        "      <td>      1193</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating</th>\n",
        "      <td>         5</td>\n",
        "      <td>         5</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>timestamp</th>\n",
        "      <td> 978300760</td>\n",
        "      <td> 978300760</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>user_id</th>\n",
        "      <td>         1</td>\n",
        "      <td>         1</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>5 rows \u00d7 2 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 83,
       "text": [
        "                   r_1        r_2\n",
        "rating_data                      \n",
        "gender               F          F\n",
        "movie_id          1193       1193\n",
        "rating               5          5\n",
        "timestamp    978300760  978300760\n",
        "user_id              1          1\n",
        "\n",
        "[5 rows x 2 columns]"
       ]
      }
     ],
     "prompt_number": 83
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.drop('r_1', axis=1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>r_2</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating_data</th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>gender</th>\n",
        "      <td>         F</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>genres</th>\n",
        "      <td>     Drama</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>movie_id</th>\n",
        "      <td>      1193</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating</th>\n",
        "      <td>         5</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>timestamp</th>\n",
        "      <td> 978300760</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>user_id</th>\n",
        "      <td>         1</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>6 rows \u00d7 1 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 84,
       "text": [
        "                   r_2\n",
        "rating_data           \n",
        "gender               F\n",
        "genres           Drama\n",
        "movie_id          1193\n",
        "rating               5\n",
        "timestamp    978300760\n",
        "user_id              1\n",
        "\n",
        "[6 rows x 1 columns]"
       ]
      }
     ],
     "prompt_number": 84
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can also delete in place with del:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "del df['r_2']\n",
      "df"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>r_1</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating_data</th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>gender</th>\n",
        "      <td>         F</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>genres</th>\n",
        "      <td>     Drama</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>movie_id</th>\n",
        "      <td>      1193</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating</th>\n",
        "      <td>         5</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>timestamp</th>\n",
        "      <td> 978300760</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>user_id</th>\n",
        "      <td>         1</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>6 rows \u00d7 1 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 85,
       "text": [
        "                   r_1\n",
        "rating_data           \n",
        "gender               F\n",
        "genres           Drama\n",
        "movie_id          1193\n",
        "rating               5\n",
        "timestamp    978300760\n",
        "user_id              1\n",
        "\n",
        "[6 rows x 1 columns]"
       ]
      }
     ],
     "prompt_number": 85
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# careful with the order here\n",
      "df['r_3'] = ['F', 'Drama', 1193, 5, 978300760, 1]\n",
      "df"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>r_1</th>\n",
        "      <th>r_3</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating_data</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>gender</th>\n",
        "      <td>         F</td>\n",
        "      <td>         F</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>genres</th>\n",
        "      <td>     Drama</td>\n",
        "      <td>     Drama</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>movie_id</th>\n",
        "      <td>      1193</td>\n",
        "      <td>      1193</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating</th>\n",
        "      <td>         5</td>\n",
        "      <td>         5</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>timestamp</th>\n",
        "      <td> 978300760</td>\n",
        "      <td> 978300760</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>user_id</th>\n",
        "      <td>         1</td>\n",
        "      <td>         1</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>6 rows \u00d7 2 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 86,
       "text": [
        "                   r_1        r_3\n",
        "rating_data                      \n",
        "gender               F          F\n",
        "genres           Drama      Drama\n",
        "movie_id          1193       1193\n",
        "rating               5          5\n",
        "timestamp    978300760  978300760\n",
        "user_id              1          1\n",
        "\n",
        "[6 rows x 2 columns]"
       ]
      }
     ],
     "prompt_number": 86
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df['r_3'] = pd.Series({'gender': 'F'})\n",
      "df"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>r_1</th>\n",
        "      <th>r_3</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating_data</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>gender</th>\n",
        "      <td>         F</td>\n",
        "      <td>   F</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>genres</th>\n",
        "      <td>     Drama</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>movie_id</th>\n",
        "      <td>      1193</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>rating</th>\n",
        "      <td>         5</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>timestamp</th>\n",
        "      <td> 978300760</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>user_id</th>\n",
        "      <td>         1</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>6 rows \u00d7 2 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 87,
       "text": [
        "                   r_1  r_3\n",
        "rating_data                \n",
        "gender               F    F\n",
        "genres           Drama  NaN\n",
        "movie_id          1193  NaN\n",
        "rating               5  NaN\n",
        "timestamp    978300760  NaN\n",
        "user_id              1  NaN\n",
        "\n",
        "[6 rows x 2 columns]"
       ]
      }
     ],
     "prompt_number": 87
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "--> Go to \"Pandas questions: Series and DataFrames\""
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### DataFrame indexing"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can index into a column using it's label, or with dot notation"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df = pd.DataFrame(data={'col_1': [0.12, 7, 45, 10], 'col_2': [0.9, 9, 34, 11]},\n",
      "                  columns=['col_1', 'col_2', 'col_3'],\n",
      "                  index=['obs1', 'obs2', 'obs3', 'obs4'])\n",
      "df"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>col_1</th>\n",
        "      <th>col_2</th>\n",
        "      <th>col_3</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>obs1</th>\n",
        "      <td>  0.12</td>\n",
        "      <td>  0.9</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs2</th>\n",
        "      <td>  7.00</td>\n",
        "      <td>  9.0</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs3</th>\n",
        "      <td> 45.00</td>\n",
        "      <td> 34.0</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs4</th>\n",
        "      <td> 10.00</td>\n",
        "      <td> 11.0</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>4 rows \u00d7 3 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 88,
       "text": [
        "      col_1  col_2 col_3\n",
        "obs1   0.12    0.9   NaN\n",
        "obs2   7.00    9.0   NaN\n",
        "obs3  45.00   34.0   NaN\n",
        "obs4  10.00   11.0   NaN\n",
        "\n",
        "[4 rows x 3 columns]"
       ]
      }
     ],
     "prompt_number": 88
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df['col_1']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 89,
       "text": [
        "obs1     0.12\n",
        "obs2     7.00\n",
        "obs3    45.00\n",
        "obs4    10.00\n",
        "Name: col_1, dtype: float64"
       ]
      }
     ],
     "prompt_number": 89
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.col_1"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 90,
       "text": [
        "obs1     0.12\n",
        "obs2     7.00\n",
        "obs3    45.00\n",
        "obs4    10.00\n",
        "Name: col_1, dtype: float64"
       ]
      }
     ],
     "prompt_number": 90
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can also use multiple columns to select a subset of them:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df[['col_2', 'col_1']]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>col_2</th>\n",
        "      <th>col_1</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>obs1</th>\n",
        "      <td>  0.9</td>\n",
        "      <td>  0.12</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs2</th>\n",
        "      <td>  9.0</td>\n",
        "      <td>  7.00</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs3</th>\n",
        "      <td> 34.0</td>\n",
        "      <td> 45.00</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs4</th>\n",
        "      <td> 11.0</td>\n",
        "      <td> 10.00</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>4 rows \u00d7 2 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 91,
       "text": [
        "      col_2  col_1\n",
        "obs1    0.9   0.12\n",
        "obs2    9.0   7.00\n",
        "obs3   34.0  45.00\n",
        "obs4   11.0  10.00\n",
        "\n",
        "[4 rows x 2 columns]"
       ]
      }
     ],
     "prompt_number": 91
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "DataFrame has similar .loc and .iloc methods:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.loc['obs1', 'col_1']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 92,
       "text": [
        "0.12"
       ]
      }
     ],
     "prompt_number": 92
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.iloc[0, 0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 93,
       "text": [
        "0.12"
       ]
      }
     ],
     "prompt_number": 93
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The .ix method gives you the most flexibility to index into certain rows, or\n",
      "even rows and columns:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.ix['obs3']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 94,
       "text": [
        "col_1     45\n",
        "col_2     34\n",
        "col_3    NaN\n",
        "Name: obs3, dtype: object"
       ]
      }
     ],
     "prompt_number": 94
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.ix[0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 95,
       "text": [
        "col_1    0.12\n",
        "col_2     0.9\n",
        "col_3     NaN\n",
        "Name: obs1, dtype: object"
       ]
      }
     ],
     "prompt_number": 95
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.ix[:2]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>col_1</th>\n",
        "      <th>col_2</th>\n",
        "      <th>col_3</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>obs1</th>\n",
        "      <td> 0.12</td>\n",
        "      <td> 0.9</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs2</th>\n",
        "      <td> 7.00</td>\n",
        "      <td> 9.0</td>\n",
        "      <td> NaN</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>2 rows \u00d7 3 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 96,
       "text": [
        "      col_1  col_2 col_3\n",
        "obs1   0.12    0.9   NaN\n",
        "obs2   7.00    9.0   NaN\n",
        "\n",
        "[2 rows x 3 columns]"
       ]
      }
     ],
     "prompt_number": 96
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.ix[:2, 'col_2']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 97,
       "text": [
        "obs1    0.9\n",
        "obs2    9.0\n",
        "Name: col_2, dtype: float64"
       ]
      }
     ],
     "prompt_number": 97
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "df.ix[:2, ['col_1', 'col_2']]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>col_1</th>\n",
        "      <th>col_2</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>obs1</th>\n",
        "      <td> 0.12</td>\n",
        "      <td> 0.9</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>obs2</th>\n",
        "      <td> 7.00</td>\n",
        "      <td> 9.0</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>2 rows \u00d7 2 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 98,
       "text": [
        "      col_1  col_2\n",
        "obs1   0.12    0.9\n",
        "obs2   7.00    9.0\n",
        "\n",
        "[2 rows x 2 columns]"
       ]
      }
     ],
     "prompt_number": 98
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "--> Go to \"Pandas questions: Indexing\""
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Break!!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## The MovieLens dataset: loading and first look\n",
      "\n",
      "Loading of the MovieLens dataset is based on the intro chapter of 'Python\n",
      "for Data Analysis\".\n",
      "\n",
      "The MovieLens data is spread across three files. We'll load each file using the `pd.read_table` function:\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "users = pd.read_table('data/ml-1m/users.dat',\n",
      "                      sep='::', header=None, \n",
      "                      names=['user_id', 'gender', 'age', 'occupation', 'zip'])\n",
      "\n",
      "ratings = pd.read_table('data/ml-1m/ratings.dat',\n",
      "                        sep='::', header=None, \n",
      "                        names=['user_id', 'movie_id', 'rating', 'timestamp'])\n",
      "\n",
      "movies = pd.read_table('data/ml-1m/movies.dat',\n",
      "                       sep='::', header=None, \n",
      "                       names=['movie_id', 'title', 'genres'])\n",
      "\n",
      "# show how one of them looks\n",
      "ratings.head(5)\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>user_id</th>\n",
        "      <th>movie_id</th>\n",
        "      <th>rating</th>\n",
        "      <th>timestamp</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>0</th>\n",
        "      <td> 1</td>\n",
        "      <td> 1193</td>\n",
        "      <td> 5</td>\n",
        "      <td> 978300760</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
        "      <td> 1</td>\n",
        "      <td>  661</td>\n",
        "      <td> 3</td>\n",
        "      <td> 978302109</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
        "      <td> 1</td>\n",
        "      <td>  914</td>\n",
        "      <td> 3</td>\n",
        "      <td> 978301968</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>3</th>\n",
        "      <td> 1</td>\n",
        "      <td> 3408</td>\n",
        "      <td> 4</td>\n",
        "      <td> 978300275</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>4</th>\n",
        "      <td> 1</td>\n",
        "      <td> 2355</td>\n",
        "      <td> 5</td>\n",
        "      <td> 978824291</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>5 rows \u00d7 4 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 99,
       "text": [
        "   user_id  movie_id  rating  timestamp\n",
        "0        1      1193       5  978300760\n",
        "1        1       661       3  978302109\n",
        "2        1       914       3  978301968\n",
        "3        1      3408       4  978300275\n",
        "4        1      2355       5  978824291\n",
        "\n",
        "[5 rows x 4 columns]"
       ]
      }
     ],
     "prompt_number": 99
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Using `pd.merge` we get it all into  one big DataFrame."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "movielens = pd.merge(pd.merge(ratings, users), movies)\n",
      "movielens.head()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>user_id</th>\n",
        "      <th>movie_id</th>\n",
        "      <th>rating</th>\n",
        "      <th>timestamp</th>\n",
        "      <th>gender</th>\n",
        "      <th>age</th>\n",
        "      <th>occupation</th>\n",
        "      <th>zip</th>\n",
        "      <th>title</th>\n",
        "      <th>genres</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>0</th>\n",
        "      <td>  1</td>\n",
        "      <td> 1193</td>\n",
        "      <td> 5</td>\n",
        "      <td> 978300760</td>\n",
        "      <td> F</td>\n",
        "      <td>  1</td>\n",
        "      <td> 10</td>\n",
        "      <td> 48067</td>\n",
        "      <td> One Flew Over the Cuckoo's Nest (1975)</td>\n",
        "      <td> Drama</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
        "      <td>  2</td>\n",
        "      <td> 1193</td>\n",
        "      <td> 5</td>\n",
        "      <td> 978298413</td>\n",
        "      <td> M</td>\n",
        "      <td> 56</td>\n",
        "      <td> 16</td>\n",
        "      <td> 70072</td>\n",
        "      <td> One Flew Over the Cuckoo's Nest (1975)</td>\n",
        "      <td> Drama</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
        "      <td> 12</td>\n",
        "      <td> 1193</td>\n",
        "      <td> 4</td>\n",
        "      <td> 978220179</td>\n",
        "      <td> M</td>\n",
        "      <td> 25</td>\n",
        "      <td> 12</td>\n",
        "      <td> 32793</td>\n",
        "      <td> One Flew Over the Cuckoo's Nest (1975)</td>\n",
        "      <td> Drama</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>3</th>\n",
        "      <td> 15</td>\n",
        "      <td> 1193</td>\n",
        "      <td> 4</td>\n",
        "      <td> 978199279</td>\n",
        "      <td> M</td>\n",
        "      <td> 25</td>\n",
        "      <td>  7</td>\n",
        "      <td> 22903</td>\n",
        "      <td> One Flew Over the Cuckoo's Nest (1975)</td>\n",
        "      <td> Drama</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>4</th>\n",
        "      <td> 17</td>\n",
        "      <td> 1193</td>\n",
        "      <td> 5</td>\n",
        "      <td> 978158471</td>\n",
        "      <td> M</td>\n",
        "      <td> 50</td>\n",
        "      <td>  1</td>\n",
        "      <td> 95350</td>\n",
        "      <td> One Flew Over the Cuckoo's Nest (1975)</td>\n",
        "      <td> Drama</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>5 rows \u00d7 10 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 100,
       "text": [
        "   user_id  movie_id  rating  timestamp gender  age  occupation    zip  \\\n",
        "0        1      1193       5  978300760      F    1          10  48067   \n",
        "1        2      1193       5  978298413      M   56          16  70072   \n",
        "2       12      1193       4  978220179      M   25          12  32793   \n",
        "3       15      1193       4  978199279      M   25           7  22903   \n",
        "4       17      1193       5  978158471      M   50           1  95350   \n",
        "\n",
        "                                    title genres  \n",
        "0  One Flew Over the Cuckoo's Nest (1975)  Drama  \n",
        "1  One Flew Over the Cuckoo's Nest (1975)  Drama  \n",
        "2  One Flew Over the Cuckoo's Nest (1975)  Drama  \n",
        "3  One Flew Over the Cuckoo's Nest (1975)  Drama  \n",
        "4  One Flew Over the Cuckoo's Nest (1975)  Drama  \n",
        "\n",
        "[5 rows x 10 columns]"
       ]
      }
     ],
     "prompt_number": 100
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Evaluation\n",
      "\n",
      "Before we start building our minimal reco engine we need a basic mechanism to evaluate the performance of our engine. For that we will:\n",
      "\n",
      "- split the data into train and test sets\n",
      "- introduce a performance criterion\n",
      "- write an `evaluate` function."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Evaluation: split ratings into train and test sets\n",
      "\n",
      "This subsection will generate training and testing sets for evaluation. You do\n",
      "not need to understand every single line of code, just the general gist:\n",
      "\n",
      "- take a smaller sample from the full 1M dataset for speed reasons;\n",
      "- make sure that we have at least 2 ratings per user in that subset;\n",
      "- split the result into training and testing sets."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# let's work with a smaller subset for speed reasons\n",
      "movielens = movielens.ix[np.random.choice(movielens.index, size=10000, replace=False)]\n",
      "print movielens.shape\n",
      "print movielens.user_id.nunique()\n",
      "print movielens.movie_id.nunique()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "(10000, 10)\n",
        "3677\n",
        "2279\n"
       ]
      }
     ],
     "prompt_number": 101
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "user_ids_larger_1 = pd.value_counts(movielens.user_id, sort=False) > 1\n",
      "user_ids_larger_1 = user_ids_larger_1[user_ids_larger_1].index\n",
      "\n",
      "movielens = movielens.select(lambda l: movielens.loc[l, 'user_id'] in user_ids_larger_1)\n",
      "print movielens.shape\n",
      "assert np.all(movielens.user_id.value_counts() > 1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "(8512, 10)\n"
       ]
      }
     ],
     "prompt_number": 102
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We now generate train and test subsets by marking 20% of each users's ratings, using groupby and apply."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def assign_to_set(df):\n",
      "    sampled_ids = np.random.choice(df.index,\n",
      "                                   size=np.int64(np.ceil(df.index.size * 0.2)),\n",
      "                                   replace=False)\n",
      "    df.ix[sampled_ids, 'for_testing'] = True\n",
      "    return df\n",
      "\n",
      "movielens['for_testing'] = False\n",
      "grouped = movielens.groupby('user_id', group_keys=False).apply(assign_to_set)\n",
      "movielens_train = movielens[grouped.for_testing == False]\n",
      "movielens_test = movielens[grouped.for_testing == True]\n",
      "print movielens.shape\n",
      "print movielens_train.shape\n",
      "print movielens_test.shape\n",
      "assert len(movielens_train.index & movielens_test.index) == 0"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "(8512, 11)\n",
        "(5845, 11)\n",
        "(2667, 11)\n"
       ]
      }
     ],
     "prompt_number": 103
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "    Store these two sets in text files:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "movielens_train.to_csv('data/my_generated_movielens_train.csv')\n",
      "movielens_test.to_csv('data/my_generated_movielens_test.csv')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 104
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Evaluation: performance criterion\n",
      "\n",
      "Performance evaluation of recommendation systems is an entire topic all in\n",
      "itself. Some of the options include:\n",
      "\n",
      "- RMSE: $\\sqrt{\\frac{\\sum(\\hat y - y)^2}{n}}$\n",
      "- Precision / Recall/ F-scores\n",
      "- ROC curves\n",
      "- Cost curves"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def compute_rmse(y_pred, y_true):\n",
      "    \"\"\" Compute Root Mean Squared Error. \"\"\"\n",
      "    \n",
      "    return np.sqrt(np.mean(np.power(y_pred - y_true, 2)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 105
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Evaluation: the 'evaluate' method"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def evaluate(estimate_f):\n",
      "    \"\"\" RMSE-based predictive performance evaluation with pandas. \"\"\"\n",
      "    \n",
      "    ids_to_estimate = zip(movielens_test.user_id, movielens_test.movie_id)\n",
      "    estimated = np.array([estimate_f(u,i) for (u,i) in ids_to_estimate])\n",
      "    real = movielens_test.rating.values\n",
      "    return compute_rmse(estimated, real)\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 106
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Minimal reco engine v1.0: simple mean ratings"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Content-based filtering using mean ratings\n",
      "\n",
      "With this table-like representation of the ratings data, a basic content-based\n",
      "filter becomes a one-liner function."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def estimate1(user_id, movie_id):\n",
      "    \"\"\" Simple content-filtering based on mean ratings. \"\"\"\n",
      "    \n",
      "    user_condition = movielens_train.user_id == user_id\n",
      "    return movielens_train.loc[user_condition, 'rating'].mean()\n",
      "\n",
      "print 'RMSE for estimate1: %s' % evaluate(estimate1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "RMSE for estimate1: 1.2459605121\n"
       ]
      }
     ],
     "prompt_number": 107
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Collaborative-based filtering using mean ratings"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def estimate2(user_id, movie_id):\n",
      "    \"\"\" Simple collaborative filter based on mean ratings. \"\"\"\n",
      "    \n",
      "    user_condition = movielens_train.user_id != user_id\n",
      "    movie_condition = movielens_train.movie_id == movie_id\n",
      "    ratings_by_others = movielens_train.loc[user_condition & movie_condition]\n",
      "    if ratings_by_others.empty: \n",
      "        return 3.0\n",
      "    else:\n",
      "        return ratings_by_others.rating.mean()\n",
      "\n",
      "print 'RMSE for estimate2: %s' % evaluate(estimate2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "RMSE for estimate2: 1.13464798007\n"
       ]
      }
     ],
     "prompt_number": 108
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "--> Go to \"Reco systems questions: Data Loading + Estimation Functions\""
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## More formulas!\n",
      "\n",
      "Here are some basic ways in which we can generalize the simple mean-based algorithms we discussed before. "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Generalizations of the aggregation function for content-based filtering: incorporating similarities\n",
      "\n",
      "Possibly incorporating metadata about items, which makes the term 'content' make more sense now.\n",
      "\n",
      "$$ r_{u,i} = k \\sum_{i' \\in I(u)} sim(i, i') \\; r_{u,i'} $$\n",
      "\n",
      "$$ r_{u,i} = \\bar r_u + k \\sum_{i' \\in I(u)} sim(i, i') \\; (r_{u,i'} - \\bar r_u) $$\n",
      "\n",
      "Here $k$ is a normalizing factor,\n",
      "\n",
      "$$ k = \\frac{1}{\\sum_{i' \\in I(u)} |sim(i,i')|} $$\n",
      "\n",
      "and $\\bar r_u$ is the average rating of user u:\n",
      "\n",
      "$$ \\bar r_u = \\frac{\\sum_{i \\in I(u)} r_{u,i}}{|I(u)|} $$\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Generalizations of the aggregation function for collaborative filtering: incorporating similarities\n",
      "\n",
      "Possibly incorporating metadata about users.\n",
      "\n",
      "$$ r_{u,i} = k \\sum_{u' \\in U(i)} sim(u, u') \\; r_{u',i} $$\n",
      "\n",
      "$$ r_{u,i} = \\bar r_u + k \\sum_{u' \\in U(i)} sim(u, u') \\; (r_{u',i} - \\bar r_u) $$\n",
      "\n",
      "Here $k$ is a normalizing factor,\n",
      "\n",
      "$$ k = \\frac{1}{\\sum_{u' \\in U(i)} |sim(u,u')|} $$\n",
      "\n",
      "and $\\bar r_u$ is the average rating of user u:\n",
      "\n",
      "$$ \\bar r_u = \\frac{\\sum_{i \\in I(u)} r_{u,i}}{|I(u)|} $$"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Aggregation in pandas"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Groupby\n",
      "\n",
      "The idea of groupby is that of *split-apply-combine*:\n",
      "\n",
      "- split data in an object according to a given key;\n",
      "- apply a function to each subset;\n",
      "- combine results into a new object."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "movielens_train.groupby('gender')['rating'].mean()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 109,
       "text": [
        "gender\n",
        "F         3.58\n",
        "M         3.53\n",
        "Name: rating, dtype: float64"
       ]
      }
     ],
     "prompt_number": 109
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "movielens_train.groupby(['gender', 'age'])['rating'].mean()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 110,
       "text": [
        "gender  age\n",
        "F       1      3.54\n",
        "        18     3.46\n",
        "        25     3.59\n",
        "        35     3.63\n",
        "        45     3.47\n",
        "        50     3.75\n",
        "        56     3.74\n",
        "M       1      3.33\n",
        "        18     3.50\n",
        "        25     3.46\n",
        "        35     3.60\n",
        "        45     3.56\n",
        "        50     3.71\n",
        "        56     3.88\n",
        "Name: rating, dtype: float64"
       ]
      }
     ],
     "prompt_number": 110
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Pivoting\n",
      "\n",
      "Let's start with a simple pivoting example that does not involve any\n",
      "aggregation. We can extract a ratings matrix as follows:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# transform the ratings frame into a ratings matrix\n",
      "ratings_mtx_df = movielens_train.pivot_table(values='rating',\n",
      "                                             rows='user_id',\n",
      "                                             cols='movie_id')\n",
      "ratings_mtx_df.head(3)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th>movie_id</th>\n",
        "      <th>1</th>\n",
        "      <th>2</th>\n",
        "      <th>3</th>\n",
        "      <th>4</th>\n",
        "      <th>5</th>\n",
        "      <th>6</th>\n",
        "      <th>8</th>\n",
        "      <th>10</th>\n",
        "      <th>11</th>\n",
        "      <th>16</th>\n",
        "      <th>17</th>\n",
        "      <th>18</th>\n",
        "      <th>19</th>\n",
        "      <th>20</th>\n",
        "      <th>21</th>\n",
        "      <th>22</th>\n",
        "      <th>23</th>\n",
        "      <th>24</th>\n",
        "      <th>25</th>\n",
        "      <th>26</th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>user_id</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>3</th>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>...</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>5</th>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>...</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>6</th>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>...</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>3 rows \u00d7 1967 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 111,
       "text": [
        "movie_id  1   2   3   4   5   6   8   10  11  16  17  18  19  20  21  22  23  \\\n",
        "user_id                                                                        \n",
        "3        NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN   \n",
        "5        NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN   \n",
        "6        NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN   \n",
        "\n",
        "movie_id  24  25  26      \n",
        "user_id                   \n",
        "3        NaN NaN NaN ...  \n",
        "5        NaN NaN NaN ...  \n",
        "6        NaN NaN NaN ...  \n",
        "\n",
        "[3 rows x 1967 columns]"
       ]
      }
     ],
     "prompt_number": 111
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# grab another subsquare of the ratings matrix to actually diplay some real entries!\n",
      "ratings_mtx_df.loc[3:11, 1196:1200]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th>movie_id</th>\n",
        "      <th>1196</th>\n",
        "      <th>1197</th>\n",
        "      <th>1198</th>\n",
        "      <th>1199</th>\n",
        "      <th>1200</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>user_id</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>3 </th>\n",
        "      <td>NaN</td>\n",
        "      <td>  5</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>5 </th>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>6 </th>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>10</th>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>11</th>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>5 rows \u00d7 5 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 193,
       "text": [
        "movie_id  1196  1197  1198  1199  1200\n",
        "user_id                               \n",
        "3          NaN     5   NaN   NaN   NaN\n",
        "5          NaN   NaN   NaN   NaN   NaN\n",
        "6          NaN   NaN   NaN   NaN   NaN\n",
        "10         NaN   NaN   NaN   NaN   NaN\n",
        "11         NaN   NaN   NaN   NaN   NaN\n",
        "\n",
        "[5 rows x 5 columns]"
       ]
      }
     ],
     "prompt_number": 193
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The more interesting case with `pivot_table` is as an interface to\n",
      "`groupby`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "movielens_train.pivot_table(values='rating', rows='age', cols='gender', aggfunc='mean')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th>gender</th>\n",
        "      <th>F</th>\n",
        "      <th>M</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>age</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>1 </th>\n",
        "      <td> 3.54</td>\n",
        "      <td> 3.33</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>18</th>\n",
        "      <td> 3.46</td>\n",
        "      <td> 3.50</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>25</th>\n",
        "      <td> 3.59</td>\n",
        "      <td> 3.46</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>35</th>\n",
        "      <td> 3.63</td>\n",
        "      <td> 3.60</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>45</th>\n",
        "      <td> 3.47</td>\n",
        "      <td> 3.56</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>50</th>\n",
        "      <td> 3.75</td>\n",
        "      <td> 3.71</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>56</th>\n",
        "      <td> 3.74</td>\n",
        "      <td> 3.88</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>7 rows \u00d7 2 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 194,
       "text": [
        "gender     F     M\n",
        "age               \n",
        "1       3.54  3.33\n",
        "18      3.46  3.50\n",
        "25      3.59  3.46\n",
        "35      3.63  3.60\n",
        "45      3.47  3.56\n",
        "50      3.75  3.71\n",
        "56      3.74  3.88\n",
        "\n",
        "[7 rows x 2 columns]"
       ]
      }
     ],
     "prompt_number": 194
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can pass in a list of functions, such as `[np.mean, np.std]`, to compute mean ratings and a measure of disagreement."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "movielens_train.pivot_table(values='rating', rows='age', cols='gender', aggfunc=[np.mean, np.std])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr>\n",
        "      <th></th>\n",
        "      <th colspan=\"2\" halign=\"left\">mean</th>\n",
        "      <th colspan=\"2\" halign=\"left\">std</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>gender</th>\n",
        "      <th>F</th>\n",
        "      <th>M</th>\n",
        "      <th>F</th>\n",
        "      <th>M</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>age</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>1 </th>\n",
        "      <td> 3.54</td>\n",
        "      <td> 3.33</td>\n",
        "      <td> 1.30</td>\n",
        "      <td> 1.24</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>18</th>\n",
        "      <td> 3.46</td>\n",
        "      <td> 3.50</td>\n",
        "      <td> 1.23</td>\n",
        "      <td> 1.16</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>25</th>\n",
        "      <td> 3.59</td>\n",
        "      <td> 3.46</td>\n",
        "      <td> 1.11</td>\n",
        "      <td> 1.13</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>35</th>\n",
        "      <td> 3.63</td>\n",
        "      <td> 3.60</td>\n",
        "      <td> 1.04</td>\n",
        "      <td> 1.04</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>45</th>\n",
        "      <td> 3.47</td>\n",
        "      <td> 3.56</td>\n",
        "      <td> 1.11</td>\n",
        "      <td> 1.04</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>50</th>\n",
        "      <td> 3.75</td>\n",
        "      <td> 3.71</td>\n",
        "      <td> 1.05</td>\n",
        "      <td> 1.07</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>56</th>\n",
        "      <td> 3.74</td>\n",
        "      <td> 3.88</td>\n",
        "      <td> 1.06</td>\n",
        "      <td> 1.06</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>7 rows \u00d7 4 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 195,
       "text": [
        "        mean         std      \n",
        "gender     F     M     F     M\n",
        "age                           \n",
        "1       3.54  3.33  1.30  1.24\n",
        "18      3.46  3.50  1.23  1.16\n",
        "25      3.59  3.46  1.11  1.13\n",
        "35      3.63  3.60  1.04  1.04\n",
        "45      3.47  3.56  1.11  1.04\n",
        "50      3.75  3.71  1.05  1.07\n",
        "56      3.74  3.88  1.06  1.06\n",
        "\n",
        "[7 rows x 4 columns]"
       ]
      }
     ],
     "prompt_number": 195
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Minimal reco engine v1.1: implicit sim functions"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We're going to need a user index from the users portion of the dataset. This will allow us to retrieve information given a specific user_id in a more convenient way:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "user_info = users.set_index('user_id')\n",
      "        user_info.head(5)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>gender</th>\n",
        "      <th>age</th>\n",
        "      <th>occupation</th>\n",
        "      <th>zip</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>user_id</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
        "      <td> F</td>\n",
        "      <td>  1</td>\n",
        "      <td> 10</td>\n",
        "      <td> 48067</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
        "      <td> M</td>\n",
        "      <td> 56</td>\n",
        "      <td> 16</td>\n",
        "      <td> 70072</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>3</th>\n",
        "      <td> M</td>\n",
        "      <td> 25</td>\n",
        "      <td> 15</td>\n",
        "      <td> 55117</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>4</th>\n",
        "      <td> M</td>\n",
        "      <td> 45</td>\n",
        "      <td>  7</td>\n",
        "      <td> 02460</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>5</th>\n",
        "      <td> M</td>\n",
        "      <td> 25</td>\n",
        "      <td> 20</td>\n",
        "      <td> 55455</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>5 rows \u00d7 4 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 196,
       "text": [
        "        gender  age  occupation    zip\n",
        "user_id                               \n",
        "1            F    1          10  48067\n",
        "2            M   56          16  70072\n",
        "3            M   25          15  55117\n",
        "4            M   45           7  02460\n",
        "5            M   25          20  55455\n",
        "\n",
        "[5 rows x 4 columns]"
       ]
      }
     ],
     "prompt_number": 196
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "With this in hand, we can now ask what the gender of a particular user_id is like so:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "user_id = 3\n",
      "user_info.loc[user_id, 'gender']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 197,
       "text": [
        "'M'"
       ]
      }
     ],
     "prompt_number": 197
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Collaborative-based filtering using implicit sim functions\n",
      "\n",
      "Using the pandas aggregation framework we will build a collaborative filter that estimates ratings using an implicit `sim(u,u')` function to compare different users."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def estimate3(user_id, movie_id):\n",
      "    \"\"\" Collaborative filtering using an implicit sim(u,u'). \"\"\"\n",
      "    \n",
      "    user_condition = movielens_train.user_id != user_id\n",
      "    movie_condition = movielens_train.movie_id == movie_id\n",
      "    ratings_by_others = movielens_train.loc[user_condition & movie_condition]\n",
      "    if ratings_by_others.empty: \n",
      "        return 3.0\n",
      "    \n",
      "    means_by_gender = ratings_by_others.pivot_table('rating', rows='movie_id', cols='gender')\n",
      "    user_gender = user_info.ix[user_id, 'gender']\n",
      "    if user_gender in means_by_gender.columns: \n",
      "        return means_by_gender.ix[movie_id, user_gender]\n",
      "    else:\n",
      "        return means_by_gender.ix[movie_id].mean()\n",
      "\n",
      "print 'RMSE for reco3: %s' % evaluate(estimate3)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "RMSE for reco3: 1.20272358698\n"
       ]
      }
     ],
     "prompt_number": 198
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "At this point it seems worthwhile to write a `learn` function to pre-compute whatever datastructures we need at estimation time."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class Reco3:\n",
      "    \"\"\" Collaborative filtering using an implicit sim(u,u'). \"\"\"\n",
      "\n",
      "    def learn(self):\n",
      "        \"\"\" Prepare datastructures for estimation. \"\"\"\n",
      "        \n",
      "        self.means_by_gender = movielens_train.pivot_table('rating', rows='movie_id', cols='gender')\n",
      "\n",
      "    def estimate(self, user_id, movie_id):\n",
      "        \"\"\" Mean ratings by other users of the same gender. \"\"\"\n",
      "        \n",
      "        if movie_id not in self.means_by_gender.index: \n",
      "            return 3.0\n",
      "        \n",
      "        user_gender = user_info.ix[user_id, 'gender']\n",
      "        if ~np.isnan(self.means_by_gender.ix[movie_id, user_gender]):\n",
      "            return self.means_by_gender.ix[movie_id, user_gender]\n",
      "        else:\n",
      "            return self.means_by_gender.ix[movie_id].mean()\n",
      "\n",
      "reco = Reco3()\n",
      "reco.learn()\n",
      "print 'RMSE for reco3: %s' % evaluate(reco.estimate)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "RMSE for reco3: 1.20272358698\n"
       ]
      }
     ],
     "prompt_number": 199
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class Reco4:\n",
      "    \"\"\" Collaborative filtering using an implicit sim(u,u'). \"\"\"\n",
      "\n",
      "    def learn(self):\n",
      "        \"\"\" Prepare datastructures for estimation. \"\"\"\n",
      "        \n",
      "        self.means_by_age = movielens_train.pivot_table('rating', rows='movie_id', cols='age')\n",
      "\n",
      "    def estimate(self, user_id, movie_id):\n",
      "        \"\"\" Mean ratings by other users of the same age. \"\"\"\n",
      "        \n",
      "        if movie_id not in self.means_by_age.index: \n",
      "            return 3.0\n",
      "        \n",
      "        user_age = user_info.ix[user_id, 'age']\n",
      "        if ~np.isnan(self.means_by_age.ix[movie_id, user_age]):\n",
      "            return self.means_by_age.ix[movie_id, user_age]\n",
      "        else:\n",
      "            return self.means_by_age.ix[movie_id].mean()\n",
      "\n",
      "reco = Reco4()\n",
      "reco.learn()\n",
      "print 'RMSE for reco4: %s' % evaluate(reco.estimate)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "RMSE for reco4: 1.21311405537\n"
       ]
      }
     ],
     "prompt_number": 200
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Mini-Challenge!\n",
      "\n",
      "- Not a real challenge\n",
      "- Focus on understanding the different versions of our minimal reco\n",
      "- Try to mix and match some of the ideas presented to come up with a minimal reco of your own\n",
      "- Evaluate it!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Mini-Challenge: first round\n",
      "Implement an `estimate` function of your own using other similarity notions, eg.:\n",
      "\n",
      "- zip code\n",
      "- movie genre\n",
      "- occupation"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Minimal reco engine v1.2: custom similarity functions"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### A few similarity functions\n",
      "\n",
      "These were all written to operate on two pandas Series, each one representing the rating history of two different users. You can also apply them to any two feature vectors that describe users or items. In all cases, the higher the return value, the more similar two Series are. You might need to add checks for edge cases, such as divisions by zero, etc.\n",
      "\n",
      "- Euclidean 'similarity'\n",
      "\n",
      "$$ sim(x,y) = \\frac{1}{1 + \\sqrt{\\sum (x - y)^2}}$$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def euclidean(s1, s2):\n",
      "    \"\"\"Take two pd.Series objects and return their euclidean 'similarity'.\"\"\"\n",
      "    diff = s1 - s2\n",
      "    return 1 / (1 + np.sqrt(np.sum(diff ** 2)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 201
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- Cosine similarity\n",
      "\n",
      "$$ sim(x,y) = \\frac{(x . y)}{\\sqrt{(x . x) (y . y)}} $$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def cosine(s1, s2):\n",
      "    \"\"\"Take two pd.Series objects and return their cosine similarity.\"\"\"\n",
      "    return np.sum(s1 * s2) / np.sqrt(np.sum(s1 ** 2) * np.sum(s2 ** 2))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 202
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- Pearson correlation\n",
      "\n",
      "$$ sim(x,y) = \\frac{(x - \\bar x).(y - \\bar y)}{\\sqrt{(x - \\bar x).(x - \\bar x) * (y - \\bar y)(y - \\bar y)}} $$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def pearson(s1, s2):\n",
      "    \"\"\"Take two pd.Series objects and return a pearson correlation.\"\"\"\n",
      "    s1_c = s1 - s1.mean()\n",
      "    s2_c = s2 - s2.mean()\n",
      "    return np.sum(s1_c * s2_c) / np.sqrt(np.sum(s1_c ** 2) * np.sum(s2_c ** 2))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 203
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- Jaccard similarity\n",
      "\n",
      "$$ sim(x,y) = \\frac{(x . y)}{(x . x) + (y . y) - (x . y)} $$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def jaccard(s1, s2):\n",
      "    dotp = np.sum(s1 * s2)\n",
      "    return dotp / (np.sum(s1 ** 2) + np.sum(s2 ** 2) - dotp)\n",
      "\n",
      "def binjaccard(s1, s2):\n",
      "    dotp = (s1.index & s2.index).size\n",
      "    return dotp / (s1.sum() + s2.sum() - dotp)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 204
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Collaborative-based filtering using custom sim functions"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class Reco5:\n",
      "    \"\"\" Collaborative filtering using a custom sim(u,u'). \"\"\"\n",
      "\n",
      "    def learn(self):\n",
      "        \"\"\" Prepare datastructures for estimation. \"\"\"\n",
      "        \n",
      "        self.all_user_profiles = movielens.pivot_table('rating', rows='movie_id', cols='user_id')\n",
      "\n",
      "    def estimate(self, user_id, movie_id):\n",
      "        \"\"\" Ratings weighted by correlation similarity. \"\"\"\n",
      "        \n",
      "        user_condition = movielens_train.user_id != user_id\n",
      "        movie_condition = movielens_train.movie_id == movie_id\n",
      "        ratings_by_others = movielens_train.loc[user_condition & movie_condition]\n",
      "        if ratings_by_others.empty: \n",
      "            return 3.0\n",
      "        \n",
      "        ratings_by_others.set_index('user_id', inplace=True)\n",
      "        their_ids = ratings_by_others.index\n",
      "        their_ratings = ratings_by_others.rating\n",
      "        their_profiles = self.all_user_profiles[their_ids]\n",
      "        user_profile = self.all_user_profiles[user_id]\n",
      "        sims = their_profiles.apply(lambda profile: pearson(profile, user_profile), axis=0)\n",
      "        ratings_sims = pd.DataFrame({'sim': sims, 'rating': their_ratings})\n",
      "        ratings_sims = ratings_sims[ratings_sims.sim > 0]\n",
      "        if ratings_sims.empty:\n",
      "            return their_ratings.mean()\n",
      "        else:\n",
      "            return np.average(ratings_sims.rating, weights=ratings_sims.sim)\n",
      "        \n",
      "reco = Reco5()\n",
      "reco.learn()\n",
      "print 'RMSE for reco5: %s' % evaluate(reco.estimate)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "RMSE for reco5: 1.08702256721\n"
       ]
      }
     ],
     "prompt_number": 205
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Mini-Challenge: second round\n",
      "Implement an `estimate` function of your own using other custom similarity notions, eg.:\n",
      "\n",
      "- euclidean\n",
      "- cosine"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## [BONUS] PyTables\n",
      "### What is it?\n",
      "\n",
      "*PyTables is a package for managing hierarchical datasets and designed to\n",
      "efficiently and easily cope with extremely large amounts of data.*\n",
      "\n",
      "### HDF5\n",
      "From [hdfgroup.org](http://www.hdfgroup.org):\n",
      "*HDF5 is a Hierarchical Data Format consisting of a data format specification and a supporting library implementation.*\n",
      "\n",
      "HDF5 files are organized in a hierarchical structure, with two primary structures: groups and datasets.\n",
      "\n",
      "- HDF5 group: a grouping structure containing instances of zero or more groups or datasets, together with supporting metadata.\n",
      "- HDF5 dataset: a multidimensional array of data elements, together with supporting metadata."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Pandas + PyTables Integration"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import tables as tb\n",
      "\n",
      "store = pd.HDFStore('data/store.h5')\n",
      "store"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 206,
       "text": [
        "<class 'pandas.io.pytables.HDFStore'>\n",
        "File path: data/store.h5\n",
        "/pycon2014/movielens            frame_table  (typ->appendable,nrows->8512,ncols->11,indexers->[index],dc->[user_id,movie_id,rating,timestamp,gender,age,occupation,zip,title,genres,for_testing])"
       ]
      }
     ],
     "prompt_number": 206
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "store.put('/pycon2014/movielens', movielens, format='table', data_columns=True)\n",
      "store"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 207,
       "text": [
        "<class 'pandas.io.pytables.HDFStore'>\n",
        "File path: data/store.h5\n",
        "/pycon2014/movielens            frame_table  (typ->appendable,nrows->8512,ncols->11,indexers->[index],dc->[user_id,movie_id,rating,timestamp,gender,age,occupation,zip,title,genres,for_testing])"
       ]
      }
     ],
     "prompt_number": 207
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "store.select('/pycon2014/movielens', \"columns=['user_id', 'rating']\", start=0, stop=5)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>user_id</th>\n",
        "      <th>rating</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>242166</th>\n",
        "      <td> 3560</td>\n",
        "      <td> 5</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>327994</th>\n",
        "      <td> 3196</td>\n",
        "      <td> 3</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>768999</th>\n",
        "      <td> 3574</td>\n",
        "      <td> 4</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>14121 </th>\n",
        "      <td> 1744</td>\n",
        "      <td> 4</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>344208</th>\n",
        "      <td> 2105</td>\n",
        "      <td> 2</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>5 rows \u00d7 2 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 208,
       "text": [
        "        user_id  rating\n",
        "242166     3560       5\n",
        "327994     3196       3\n",
        "768999     3574       4\n",
        "14121      1744       4\n",
        "344208     2105       2\n",
        "\n",
        "[5 rows x 2 columns]"
       ]
      }
     ],
     "prompt_number": 208
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "store.select('/pycon2014/movielens', where=u'rating>4', start=0, stop=20)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>user_id</th>\n",
        "      <th>movie_id</th>\n",
        "      <th>rating</th>\n",
        "      <th>timestamp</th>\n",
        "      <th>gender</th>\n",
        "      <th>age</th>\n",
        "      <th>occupation</th>\n",
        "      <th>zip</th>\n",
        "      <th>title</th>\n",
        "      <th>genres</th>\n",
        "      <th>for_testing</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>242166</th>\n",
        "      <td> 3560</td>\n",
        "      <td>  296</td>\n",
        "      <td> 5</td>\n",
        "      <td> 966796358</td>\n",
        "      <td> F</td>\n",
        "      <td> 25</td>\n",
        "      <td> 6</td>\n",
        "      <td> 74105</td>\n",
        "      <td>    Pulp Fiction (1994)</td>\n",
        "      <td>                Crime|Drama</td>\n",
        "      <td> False</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>624536</th>\n",
        "      <td> 5283</td>\n",
        "      <td> 2761</td>\n",
        "      <td> 5</td>\n",
        "      <td> 961166145</td>\n",
        "      <td> M</td>\n",
        "      <td> 18</td>\n",
        "      <td> 2</td>\n",
        "      <td> 63138</td>\n",
        "      <td> Iron Giant, The (1999)</td>\n",
        "      <td>       Animation|Children's</td>\n",
        "      <td> False</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>512865</th>\n",
        "      <td> 5767</td>\n",
        "      <td> 1200</td>\n",
        "      <td> 5</td>\n",
        "      <td> 958176105</td>\n",
        "      <td> M</td>\n",
        "      <td> 25</td>\n",
        "      <td> 2</td>\n",
        "      <td> 75287</td>\n",
        "      <td>          Aliens (1986)</td>\n",
        "      <td> Action|Sci-Fi|Thriller|War</td>\n",
        "      <td> False</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>3 rows \u00d7 11 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 209,
       "text": [
        "        user_id  movie_id  rating  timestamp gender  age  occupation    zip  \\\n",
        "242166     3560       296       5  966796358      F   25           6  74105   \n",
        "624536     5283      2761       5  961166145      M   18           2  63138   \n",
        "512865     5767      1200       5  958176105      M   25           2  75287   \n",
        "\n",
        "                         title                      genres for_testing  \n",
        "242166     Pulp Fiction (1994)                 Crime|Drama       False  \n",
        "624536  Iron Giant, The (1999)        Animation|Children's       False  \n",
        "512865           Aliens (1986)  Action|Sci-Fi|Thriller|War       False  \n",
        "\n",
        "[3 rows x 11 columns]"
       ]
      }
     ],
     "prompt_number": 209
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "store.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 212
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Direct File Manipulation: Node Attributes Example"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import tables as tb\n",
      "from datetime import datetime\n",
      "\n",
      "hdf_file = tb.open_file('data/store.h5', 'r+')\n",
      "node = hdf_file.getNode('/pycon2014/movielens')\n",
      "hdf_file.setNodeAttr(node, 'last_modified', datetime.utcnow())\n",
      "node._v_attrs"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 213,
       "text": [
        "/pycon2014/movielens._v_attrs (AttributeSet), 15 attributes:\n",
        "   [CLASS := 'GROUP',\n",
        "    TITLE := '',\n",
        "    VERSION := '1.0',\n",
        "    data_columns := ['user_id', 'movie_id', 'rating', 'timestamp', 'gender', 'age', 'occupation', 'zip', 'title', 'genres', 'for_testing'],\n",
        "    encoding := None,\n",
        "    index_cols := [(0, 'index')],\n",
        "    info := {1: {'type': 'Index', 'names': [None]}, 'index': {}},\n",
        "    last_modified := datetime.datetime(2014, 4, 7, 20, 50, 43, 588596),\n",
        "    levels := 1,\n",
        "    nan_rep := 'nan',\n",
        "    non_index_axes := [(1, ['user_id', 'movie_id', 'rating', 'timestamp', 'gender', 'age', 'occupation', 'zip', 'title', 'genres', 'for_testing'])],\n",
        "    pandas_type := 'frame_table',\n",
        "    pandas_version := '0.10.1',\n",
        "    table_type := 'appendable_frame',\n",
        "    values_cols := ['user_id', 'movie_id', 'rating', 'timestamp', 'gender', 'age', 'occupation', 'zip', 'title', 'genres', 'for_testing']]"
       ]
      }
     ],
     "prompt_number": 213
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "hdf_file.getNodeAttr(node,'last_modified')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 214,
       "text": [
        "datetime.datetime(2014, 4, 7, 20, 50, 43, 588596)"
       ]
      }
     ],
     "prompt_number": 214
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Handling things that don't fit in memory"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "group_3 = h5file.createGroup(h5file.root, 'group_3', 'Group Three')\n",
      "ndim = 6000000\n",
      "h5file.createArray(group_3, 'random_group_3',\n",
      "                   numpy.zeros((ndim,ndim)), \"A very very large array\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rows = 10\n",
      "cols = 10\n",
      "earr = h5file.createEArray(group_3, 'EArray', tb.Int8Atom(),\n",
      "                           (0, cols), \"A very very large array, second try.\")\n",
      "\n",
      "for i in range(rows):\n",
      "    earr.append(numpy.zeros((1, cols)))\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "earr"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## References and further reading\n",
      "\n",
      "- Goldberg, D., D. Nichols, B. M. Oki, and D. Terry. \u201cUsing Collaborative Filtering to Weave an Information Tapestry.\u201d Communications of the ACM 35, no. 12 (1992): 61\u201370.\n",
      "- Resnick, Paul, and Hal R. Varian. \u201cRecommender Systems.\u201d Commun. ACM 40, no. 3 (March 1997): 56\u201358. doi:10.1145/245108.245121.\n",
      "- Adomavicius, Gediminas, and Alexander Tuzhilin. \u201cToward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions.\u201d IEEE Transactions on Knowledge and Data Engineering 17, no. 6 (2005): 734\u2013749. doi:http://doi.ieeecomputersociety.org/10.1109/TKDE.2005.99.\n",
      "- Adomavicius, Gediminas, Ramesh Sankaranarayanan, Shahana Sen, and Alexander Tuzhilin. \u201cIncorporating Contextual Information in Recommender Systems Using a Multidimensional Approach.\u201d ACM Trans. Inf. Syst. 23, no. 1 (2005): 103\u2013145. doi:10.1145/1055709.1055714.\n",
      "- Koren, Y., R. Bell, and C. Volinsky. \u201cMatrix Factorization Techniques for Recommender Systems.\u201d Computer 42, no. 8 (2009): 30\u201337.\n",
      "- William Wesley McKinney. Python for Data Analysis. O\u2019Reilly, 2012.\n",
      "- Toby Segaran. Programming Collective Intelligence. O\u2019Reilly, 2007.\n",
      "- Zhou, Tao, Zoltan Kuscsik, Jian-Guo Liu, Matus Medo, Joseph R Wakeling, and Yi-Cheng Zhang. \u201cSolving the Apparent Diversity-accuracy Dilemma of Recommender Systems.\u201d arXiv:0808.2670 (August 19, 2008). doi:10.1073/pnas.1000488107.\n",
      "- Shani, G., D. Heckerman, and R. I Brafman. \u201cAn MDP-based Recommender System.\u201d Journal of Machine Learning Research 6, no. 2 (2006): 1265.\n",
      "- Joseph A. Konstan, John Riedl. \"Deconstructing Recommender Systems.\" IEEE Spectrum, October 2012."
     ]
    }
   ],
   "metadata": {}
  }
 ]
}